[PATCH 0/4] iommu/vt-d: Disable igfx iommu superpage on bxt/skl/glk

[PATCH 0/4] iommu/vt-d: Disable igfx iommu superpage on bxt/skl/glk

[Ville Syrjala]

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

I ran into some kind of fail with VT-d superpage on Geminlake igfx,
so without any better ideas let's just disable it.

Additionally Skylake/Broxton igfx have known issues with VT-d
superpage as well, so let's disable it there as well. This should
let us re-enable frame buffer compression (FBC) for some extra
power savings when the display is on.

Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org

Ville Syrjälä (4):
  iommu/vt-d: Disable superpage for Geminilake igfx
  iommu/vt-d: Disable superpage for Broxton igfx
  iommu/vt-d: Disable superpage for Skylake igfx
  drm/i915/fbc: Allow FBC + VT-d on SKL/BXT

 drivers/gpu/drm/i915/display/intel_fbc.c | 16 ---------
 drivers/iommu/intel/iommu.c              | 44 ++++++++++++++++++++++++
 2 files changed, 44 insertions(+), 16 deletions(-)

-- 
2.31.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Ville Syrjala]

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

While running "gem_exec_big --r single" from igt-gpu-tools on
Geminilake as soon as a 2M mapping is made I tend to get a DMAR
write fault. Strangely the faulting address is always a 4K page
and usually very far away from the 2M page that got mapped.
But if no 2M mappings get used I can't reproduce the fault.

I also tried to dump the PTE for the faulting address but it actually
looks correct to me (ie. definitely seems to have the write bit set):
 DMAR: DRHD: handling fault status reg 2
 DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
 DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003

So not really sure what's going on and this might just be full on duct
tape, but it seems to work here. The machine has now survived a whole day
running that test whereas with superpage enabled it fails in less than
a minute usually.

TODO: might be nice to disable superpage only for the igfx iommu
      instead of both iommus
TODO: would be nice to use the macros from include/drm/i915_pciids.h,
      but can't do that with DECLARE_PCI_FIXUP_HEADER()

Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 19c7888cbb86..4fff2c9c86af 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -5617,6 +5617,16 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1632, quirk_iommu_igfx);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
 
+static void quirk_iommu_nosp(struct pci_dev *dev)
+{
+	pci_info(dev, "Disabling IOMMU superpage for graphics on this chipset\n");
+	intel_iommu_superpage = 0;
+}
+
+/* Geminilake igfx appears to have issues with superpage */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3185, quirk_iommu_nosp);
+
 static void quirk_iommu_rwbf(struct pci_dev *dev)
 {
 	if (risky_device(dev))
-- 
2.31.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH 2/4] iommu/vt-d: Disable superpage for Broxton igfx

[Ville Syrjala]

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

Broxton has known issues with VT-d superpage. Namely frame buffer
compression (FBC) can't be safely used when superpage is enabled.
Currently we're disabling FBC entirely when VT-d is active, but
I think just disabling superpage would be better since FBC can
save some power.

TODO: might be nice to disable superpage only for the igfx iommu
      instead of both iommus
TODO: would be nice to use the macros from include/drm/i915_pciids.h,
      but can't do that with DECLARE_PCI_FIXUP_HEADER()

Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 4fff2c9c86af..40117f868761 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -5623,6 +5623,13 @@ static void quirk_iommu_nosp(struct pci_dev *dev)
 	intel_iommu_superpage = 0;
 }
 
+/* Broxton igfx has issues with superpage */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0A84, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1A84, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1A85, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x5A84, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x5A85, quirk_iommu_nosp);
+
 /* Geminilake igfx appears to have issues with superpage */
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, quirk_iommu_nosp);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3185, quirk_iommu_nosp);
-- 
2.31.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH 3/4] iommu/vt-d: Disable superpage for Skylake igfx

[Ville Syrjala]

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

Skylake has known issues with VT-d superpage. Namely frame buffer
compression (FBC) can't be safely used when superpage is enabled.
Currently we're disabling FBC entirely when VT-d is active, but
I think just disabling superpage would be better since FBC can
save some power.

TODO: might be nice to disable superpage only for the igfx iommu
      instead of both iommus
TODO: would be nice to use the macros from include/drm/i915_pciids.h,
      but can't do that with DECLARE_PCI_FIXUP_HEADER()

Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
 drivers/iommu/intel/iommu.c | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 40117f868761..14f951ca4799 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -5623,6 +5623,33 @@ static void quirk_iommu_nosp(struct pci_dev *dev)
 	intel_iommu_superpage = 0;
 }
 
+/* Skylake igfx has issues with superpage */
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1906, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1913, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x190E, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1915, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1902, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x190A, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x190B, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1917, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1916, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1921, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x191E, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1912, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x191A, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x191B, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x191D, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1923, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1926, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1927, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x192A, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x192B, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x192D, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1932, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x193A, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x193B, quirk_iommu_nosp);
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x193D, quirk_iommu_nosp);
+
 /* Broxton igfx has issues with superpage */
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0A84, quirk_iommu_nosp);
 DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1A84, quirk_iommu_nosp);
-- 
2.31.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

[PATCH 4/4] drm/i915/fbc: Allow FBC + VT-d on SKL/BXT

[Ville Syrjala]

From: Ville Syrjälä <ville.syrjala@linux.intel.com>

With the iommu driver disabling VT-d superpage it should be
safe to use FBC on SKL/BXT with VT-d otherwise enabled.

Cc: David Woodhouse <dwmw2@infradead.org>
Cc: Lu Baolu <baolu.lu@linux.intel.com>
Cc: iommu@lists.linux-foundation.org
Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
---
 drivers/gpu/drm/i915/display/intel_fbc.c | 16 ----------------
 1 file changed, 16 deletions(-)

diff --git a/drivers/gpu/drm/i915/display/intel_fbc.c b/drivers/gpu/drm/i915/display/intel_fbc.c
index 82effb64a3b9..de44f93a33d0 100644
--- a/drivers/gpu/drm/i915/display/intel_fbc.c
+++ b/drivers/gpu/drm/i915/display/intel_fbc.c
@@ -1448,19 +1448,6 @@ static int intel_sanitize_fbc_option(struct drm_i915_private *dev_priv)
 	return 0;
 }
 
-static bool need_fbc_vtd_wa(struct drm_i915_private *dev_priv)
-{
-	/* WaFbcTurnOffFbcWhenHyperVisorIsUsed:skl,bxt */
-	if (intel_vtd_active() &&
-	    (IS_SKYLAKE(dev_priv) || IS_BROXTON(dev_priv))) {
-		drm_info(&dev_priv->drm,
-			 "Disabling framebuffer compression (FBC) to prevent screen flicker with VT-d enabled\n");
-		return true;
-	}
-
-	return false;
-}
-
 /**
  * intel_fbc_init - Initialize FBC
  * @dev_priv: the i915 device
@@ -1478,9 +1465,6 @@ void intel_fbc_init(struct drm_i915_private *dev_priv)
 	if (!drm_mm_initialized(&dev_priv->mm.stolen))
 		mkwrite_device_info(dev_priv)->display.has_fbc = false;
 
-	if (need_fbc_vtd_wa(dev_priv))
-		mkwrite_device_info(dev_priv)->display.has_fbc = false;
-
 	dev_priv->params.enable_fbc = intel_sanitize_fbc_option(dev_priv);
 	drm_dbg_kms(&dev_priv->drm, "Sanitized enable_fbc value: %d\n",
 		    dev_priv->params.enable_fbc);
-- 
2.31.1

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Lu Baolu]

On 7/10/21 12:47 AM, Ville Syrjala wrote:
> From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> 
> While running "gem_exec_big --r single" from igt-gpu-tools on
> Geminilake as soon as a 2M mapping is made I tend to get a DMAR
> write fault. Strangely the faulting address is always a 4K page
> and usually very far away from the 2M page that got mapped.
> But if no 2M mappings get used I can't reproduce the fault.
> 
> I also tried to dump the PTE for the faulting address but it actually
> looks correct to me (ie. definitely seems to have the write bit set):
>   DMAR: DRHD: handling fault status reg 2
>   DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
>   DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
> 
> So not really sure what's going on and this might just be full on duct
> tape, but it seems to work here. The machine has now survived a whole day
> running that test whereas with superpage enabled it fails in less than
> a minute usually.
> 
> TODO: might be nice to disable superpage only for the igfx iommu
>        instead of both iommus

If all these quirks are about igfx dedicated iommu's, I would suggest to
disable superpage only for the igfx ones.

Best regards,
baolu

> TODO: would be nice to use the macros from include/drm/i915_pciids.h,
>        but can't do that with DECLARE_PCI_FIXUP_HEADER()
> 
> Cc: David Woodhouse <dwmw2@infradead.org>
> Cc: Lu Baolu <baolu.lu@linux.intel.com>
> Cc: iommu@lists.linux-foundation.org
> Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
> ---
>   drivers/iommu/intel/iommu.c | 10 ++++++++++
>   1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 19c7888cbb86..4fff2c9c86af 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -5617,6 +5617,16 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1632, quirk_iommu_igfx);
>   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
>   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
>   
> +static void quirk_iommu_nosp(struct pci_dev *dev)
> +{
> +	pci_info(dev, "Disabling IOMMU superpage for graphics on this chipset\n");
> +	intel_iommu_superpage = 0;
> +}
> +
> +/* Geminilake igfx appears to have issues with superpage */
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, quirk_iommu_nosp);
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3185, quirk_iommu_nosp);
> +
>   static void quirk_iommu_rwbf(struct pci_dev *dev)
>   {
>   	if (risky_device(dev))
> 
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Ville Syrjälä]

On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
> On 7/10/21 12:47 AM, Ville Syrjala wrote:
> > From: Ville Syrjälä <ville.syrjala@linux.intel.com>
> > 
> > While running "gem_exec_big --r single" from igt-gpu-tools on
> > Geminilake as soon as a 2M mapping is made I tend to get a DMAR
> > write fault. Strangely the faulting address is always a 4K page
> > and usually very far away from the 2M page that got mapped.
> > But if no 2M mappings get used I can't reproduce the fault.
> > 
> > I also tried to dump the PTE for the faulting address but it actually
> > looks correct to me (ie. definitely seems to have the write bit set):
> >   DMAR: DRHD: handling fault status reg 2
> >   DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
> >   DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
> > 
> > So not really sure what's going on and this might just be full on duct
> > tape, but it seems to work here. The machine has now survived a whole day
> > running that test whereas with superpage enabled it fails in less than
> > a minute usually.
> > 
> > TODO: might be nice to disable superpage only for the igfx iommu
> >        instead of both iommus
> 
> If all these quirks are about igfx dedicated iommu's, I would suggest to
> disable superpage only for the igfx ones.

Sure. Unfortunately there's no convenient mechanism to do that in
the iommu driver that I can immediately see. So not something I
can just whip up easily. Since you're actually familiar with the
driver maybe you can come up with a decent solution for that?

> 
> Best regards,
> baolu
> 
> > TODO: would be nice to use the macros from include/drm/i915_pciids.h,
> >        but can't do that with DECLARE_PCI_FIXUP_HEADER()
> > 
> > Cc: David Woodhouse <dwmw2@infradead.org>
> > Cc: Lu Baolu <baolu.lu@linux.intel.com>
> > Cc: iommu@lists.linux-foundation.org
> > Signed-off-by: Ville Syrjälä <ville.syrjala@linux.intel.com>
> > ---
> >   drivers/iommu/intel/iommu.c | 10 ++++++++++
> >   1 file changed, 10 insertions(+)
> > 
> > diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> > index 19c7888cbb86..4fff2c9c86af 100644
> > --- a/drivers/iommu/intel/iommu.c
> > +++ b/drivers/iommu/intel/iommu.c
> > @@ -5617,6 +5617,16 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x1632, quirk_iommu_igfx);
> >   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
> >   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
> >   
> > +static void quirk_iommu_nosp(struct pci_dev *dev)
> > +{
> > +	pci_info(dev, "Disabling IOMMU superpage for graphics on this chipset\n");
> > +	intel_iommu_superpage = 0;
> > +}
> > +
> > +/* Geminilake igfx appears to have issues with superpage */
> > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, quirk_iommu_nosp);
> > +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3185, quirk_iommu_nosp);
> > +
> >   static void quirk_iommu_rwbf(struct pci_dev *dev)
> >   {
> >   	if (risky_device(dev))
> > 

-- 
Ville Syrjälä
Intel
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Lu Baolu]

On 7/12/21 11:47 PM, Ville Syrjälä wrote:
> On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
>> On 7/10/21 12:47 AM, Ville Syrjala wrote:
>>> From: Ville Syrjälä<ville.syrjala@linux.intel.com>
>>>
>>> While running "gem_exec_big --r single" from igt-gpu-tools on
>>> Geminilake as soon as a 2M mapping is made I tend to get a DMAR
>>> write fault. Strangely the faulting address is always a 4K page
>>> and usually very far away from the 2M page that got mapped.
>>> But if no 2M mappings get used I can't reproduce the fault.
>>>
>>> I also tried to dump the PTE for the faulting address but it actually
>>> looks correct to me (ie. definitely seems to have the write bit set):
>>>    DMAR: DRHD: handling fault status reg 2
>>>    DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
>>>    DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
>>>
>>> So not really sure what's going on and this might just be full on duct
>>> tape, but it seems to work here. The machine has now survived a whole day
>>> running that test whereas with superpage enabled it fails in less than
>>> a minute usually.
>>>
>>> TODO: might be nice to disable superpage only for the igfx iommu
>>>         instead of both iommus
>> If all these quirks are about igfx dedicated iommu's, I would suggest to
>> disable superpage only for the igfx ones.
> Sure. Unfortunately there's no convenient mechanism to do that in
> the iommu driver that I can immediately see. So not something I
> can just whip up easily. Since you're actually familiar with the
> driver maybe you can come up with a decent solution for that?
> 

How about something like below? [no compile, no test...]

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 1131b8efb050..2d51ef288a9e 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -338,6 +338,7 @@ static int intel_iommu_strict;
  static int intel_iommu_superpage = 1;
  static int iommu_identity_mapping;
  static int iommu_skip_te_disable;
+static int iommu_skip_igfx_superpage;

  #define IDENTMAP_GFX		2
  #define IDENTMAP_AZALIA		4
@@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct 
intel_iommu *skip)
  	return ret;
  }

+static bool domain_use_super_page(struct dmar_domain *domain)
+{
+	struct dmar_drhd_unit *drhd;
+	struct intel_iommu *iommu;
+	bool ret = true;
+
+	if (!intel_iommu_superpage)
+		return false;
+
+	rcu_read_lock();
+	for_each_active_iommu(iommu, drhd) {
+		if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
+			ret = false;
+			break
+		}
+	}
+	rcu_read_unlock();
+
+	return ret;
+}
+
  static int domain_update_iommu_superpage(struct dmar_domain *domain,
  					 struct intel_iommu *skip)
  {
@@ -659,7 +681,7 @@ static int domain_update_iommu_superpage(struct 
dmar_domain *domain,
  	struct intel_iommu *iommu;
  	int mask = 0x3;

-	if (!intel_iommu_superpage)
+	if (!domain_use_super_page(domain))
  		return 0;

  	/* set iommu_superpage to the smallest common denominator */
@@ -5656,6 +5678,14 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 
0x1632, quirk_iommu_igfx);
  DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
  DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);

+static void quirk_skip_igfx_superpage(struct pci_dev *dev)
+{
+	pci_info(dev, "Disabling IOMMU superpage for graphics on this chipset\n");
+	iommu_skip_igfx_superpage = 1;
+}
+
+DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, 
quirk_skip_igfx_superpage);
+
  static void quirk_iommu_rwbf(struct pci_dev *dev)
  {
  	if (risky_device(dev))

Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Ville Syrjälä]

On Tue, Jul 13, 2021 at 09:34:09AM +0800, Lu Baolu wrote:
> On 7/12/21 11:47 PM, Ville Syrjälä wrote:
> > On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
> >> On 7/10/21 12:47 AM, Ville Syrjala wrote:
> >>> From: Ville Syrjälä<ville.syrjala@linux.intel.com>
> >>>
> >>> While running "gem_exec_big --r single" from igt-gpu-tools on
> >>> Geminilake as soon as a 2M mapping is made I tend to get a DMAR
> >>> write fault. Strangely the faulting address is always a 4K page
> >>> and usually very far away from the 2M page that got mapped.
> >>> But if no 2M mappings get used I can't reproduce the fault.
> >>>
> >>> I also tried to dump the PTE for the faulting address but it actually
> >>> looks correct to me (ie. definitely seems to have the write bit set):
> >>>    DMAR: DRHD: handling fault status reg 2
> >>>    DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
> >>>    DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
> >>>
> >>> So not really sure what's going on and this might just be full on duct
> >>> tape, but it seems to work here. The machine has now survived a whole day
> >>> running that test whereas with superpage enabled it fails in less than
> >>> a minute usually.
> >>>
> >>> TODO: might be nice to disable superpage only for the igfx iommu
> >>>         instead of both iommus
> >> If all these quirks are about igfx dedicated iommu's, I would suggest to
> >> disable superpage only for the igfx ones.
> > Sure. Unfortunately there's no convenient mechanism to do that in
> > the iommu driver that I can immediately see. So not something I
> > can just whip up easily. Since you're actually familiar with the
> > driver maybe you can come up with a decent solution for that?
> > 
> 
> How about something like below? [no compile, no test...]
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index 1131b8efb050..2d51ef288a9e 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -338,6 +338,7 @@ static int intel_iommu_strict;
>   static int intel_iommu_superpage = 1;
>   static int iommu_identity_mapping;
>   static int iommu_skip_te_disable;
> +static int iommu_skip_igfx_superpage;
> 
>   #define IDENTMAP_GFX		2
>   #define IDENTMAP_AZALIA		4
> @@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct 
> intel_iommu *skip)
>   	return ret;
>   }
> 
> +static bool domain_use_super_page(struct dmar_domain *domain)
> +{
> +	struct dmar_drhd_unit *drhd;
> +	struct intel_iommu *iommu;
> +	bool ret = true;
> +
> +	if (!intel_iommu_superpage)
> +		return false;
> +
> +	rcu_read_lock();
> +	for_each_active_iommu(iommu, drhd) {
> +		if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
> +			ret = false;
> +			break
                             ^
Missing semicolon. Othwerwise seems to work great here. Thanks.

Are you going to turn this into a proper patch, or do you
want me to just squash this into my patches and repost?

> +		}
> +	}
> +	rcu_read_unlock();
> +
> +	return ret;
> +}
> +
>   static int domain_update_iommu_superpage(struct dmar_domain *domain,
>   					 struct intel_iommu *skip)
>   {
> @@ -659,7 +681,7 @@ static int domain_update_iommu_superpage(struct 
> dmar_domain *domain,
>   	struct intel_iommu *iommu;
>   	int mask = 0x3;
> 
> -	if (!intel_iommu_superpage)
> +	if (!domain_use_super_page(domain))
>   		return 0;
> 
>   	/* set iommu_superpage to the smallest common denominator */
> @@ -5656,6 +5678,14 @@ DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 
> 0x1632, quirk_iommu_igfx);
>   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163A, quirk_iommu_igfx);
>   DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x163D, quirk_iommu_igfx);
> 
> +static void quirk_skip_igfx_superpage(struct pci_dev *dev)
> +{
> +	pci_info(dev, "Disabling IOMMU superpage for graphics on this chipset\n");
> +	iommu_skip_igfx_superpage = 1;
> +}
> +
> +DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x3184, 
> quirk_skip_igfx_superpage);
> +
>   static void quirk_iommu_rwbf(struct pci_dev *dev)
>   {
>   	if (risky_device(dev))
> 
> Best regards,
> baolu

-- 
Ville Syrjälä
Intel
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH 1/4] iommu/vt-d: Disable superpage for Geminilake igfx

[Lu Baolu]

On 7/14/21 4:30 AM, Ville Syrjälä wrote:
> On Tue, Jul 13, 2021 at 09:34:09AM +0800, Lu Baolu wrote:
>> On 7/12/21 11:47 PM, Ville Syrjälä wrote:
>>> On Mon, Jul 12, 2021 at 07:23:07AM +0800, Lu Baolu wrote:
>>>> On 7/10/21 12:47 AM, Ville Syrjala wrote:
>>>>> From: Ville Syrjälä<ville.syrjala@linux.intel.com>
>>>>>
>>>>> While running "gem_exec_big --r single" from igt-gpu-tools on
>>>>> Geminilake as soon as a 2M mapping is made I tend to get a DMAR
>>>>> write fault. Strangely the faulting address is always a 4K page
>>>>> and usually very far away from the 2M page that got mapped.
>>>>> But if no 2M mappings get used I can't reproduce the fault.
>>>>>
>>>>> I also tried to dump the PTE for the faulting address but it actually
>>>>> looks correct to me (ie. definitely seems to have the write bit set):
>>>>>     DMAR: DRHD: handling fault status reg 2
>>>>>     DMAR: [DMA Write] Request device [00:02.0] PASID ffffffff fault addr 7fa8a78000 [fault reason 05] PTE Write access is not set
>>>>>     DMAR: fault 7fa8a78000 (level=1) PTE = 149efc003
>>>>>
>>>>> So not really sure what's going on and this might just be full on duct
>>>>> tape, but it seems to work here. The machine has now survived a whole day
>>>>> running that test whereas with superpage enabled it fails in less than
>>>>> a minute usually.
>>>>>
>>>>> TODO: might be nice to disable superpage only for the igfx iommu
>>>>>          instead of both iommus
>>>> If all these quirks are about igfx dedicated iommu's, I would suggest to
>>>> disable superpage only for the igfx ones.
>>> Sure. Unfortunately there's no convenient mechanism to do that in
>>> the iommu driver that I can immediately see. So not something I
>>> can just whip up easily. Since you're actually familiar with the
>>> driver maybe you can come up with a decent solution for that?
>>>
>> How about something like below? [no compile, no test...]
>>
>> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
>> index 1131b8efb050..2d51ef288a9e 100644
>> --- a/drivers/iommu/intel/iommu.c
>> +++ b/drivers/iommu/intel/iommu.c
>> @@ -338,6 +338,7 @@ static int intel_iommu_strict;
>>    static int intel_iommu_superpage = 1;
>>    static int iommu_identity_mapping;
>>    static int iommu_skip_te_disable;
>> +static int iommu_skip_igfx_superpage;
>>
>>    #define IDENTMAP_GFX		2
>>    #define IDENTMAP_AZALIA		4
>> @@ -652,6 +653,27 @@ static bool domain_update_iommu_snooping(struct
>> intel_iommu *skip)
>>    	return ret;
>>    }
>>
>> +static bool domain_use_super_page(struct dmar_domain *domain)
>> +{
>> +	struct dmar_drhd_unit *drhd;
>> +	struct intel_iommu *iommu;
>> +	bool ret = true;
>> +
>> +	if (!intel_iommu_superpage)
>> +		return false;
>> +
>> +	rcu_read_lock();
>> +	for_each_active_iommu(iommu, drhd) {
>> +		if (drhd->gfx_dedicated && iommu_skip_igfx_superpage) {
>> +			ret = false;
>> +			break
>                               ^
> Missing semicolon. Othwerwise seems to work great here. Thanks.
> 
> Are you going to turn this into a proper patch, or do you
> want me to just squash this into my patches and repost?
> 

Please go ahead with a new version.

Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu