[PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Ting-Wei Lan]

When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
devices, It is possible to encounter graphics issues that make screen
unreadable or crash the system. It was reported in freedesktop bugzilla:

https://bugs.freedesktop.org/show_bug.cgi?id=90037

As we still cannot find a proper fix for this problem, this patch adds
iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
users to manually workaround the problem.

Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
---
 docs/misc/xen-command-line.markdown  | 10 +++++++++-
 xen/drivers/passthrough/iommu.c      |  3 +++
 xen/drivers/passthrough/vtd/quirks.c |  2 +-
 xen/include/xen/iommu.h              |  2 +-
 4 files changed, 14 insertions(+), 3 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 13f03ad..7b61603 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -793,7 +793,7 @@ debug hypervisor only).
 > Default: `new` unless directed-EOI is supported
 
 ### iommu
-> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | verbose | debug ]`
+> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | igfx_off | verbose | debug ]`
 
 > Sub-options:
 
@@ -867,6 +867,14 @@ debug hypervisor only).
 >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
 >> entry aren't PCI discoverable).
 
+> `igfx_off` (VT-d)
+
+> Default: `false`
+
+>> Workaround graphics issues for Intel Calpella/Ironlake devices. This option
+>> is similar to Linux `intel_iommu=igfx_off`, so if it fixes anything, you
+>> should file a bug reporting the problem.
+
 > `verbose`
 
 > Default: `false`
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index cc12735..2ba8c9d 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -47,6 +47,7 @@ bool_t __read_mostly force_iommu;
 bool_t __hwdom_initdata iommu_dom0_strict;
 bool_t __read_mostly iommu_verbose;
 bool_t __read_mostly iommu_workaround_bios_bug;
+bool_t __read_mostly iommu_igfx_off;
 bool_t __read_mostly iommu_passthrough;
 bool_t __read_mostly iommu_snoop = 1;
 bool_t __read_mostly iommu_qinval = 1;
@@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
             force_iommu = val;
         else if ( !strcmp(s, "workaround_bios_bug") )
             iommu_workaround_bios_bug = val;
+        else if ( !strcmp(s, "igfx_off") )
+            iommu_igfx_off = val;
         else if ( !strcmp(s, "verbose") )
             iommu_verbose = val;
         else if ( !strcmp(s, "snoop") )
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 69d29ab..da1d853 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -77,7 +77,7 @@ int is_igd_vt_enabled_quirk(void)
 
     /* integrated graphics on Intel platforms is located at 0:2.0 */
     ggc = pci_conf_read16(0, 0, IGD_DEV, 0, GGC);
-    return ( ggc & GGC_MEMORY_VT_ENABLED ? 1 : 0 );
+    return ( ggc & GGC_MEMORY_VT_ENABLED ? 1 : 0 ) && !iommu_igfx_off;
 }
 
 /*
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8eb764a..2cdb0a9 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -29,7 +29,7 @@
 
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
-extern bool_t iommu_workaround_bios_bug, iommu_passthrough;
+extern bool_t iommu_workaround_bios_bug, iommu_igfx_off, iommu_passthrough;
 extern bool_t iommu_snoop, iommu_qinval, iommu_intremap;
 extern bool_t iommu_hap_pt_share;
 extern bool_t iommu_debug;
-- 
2.4.3

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Andrew Cooper]

On 17/07/15 20:05, Ting-Wei Lan wrote:
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
>
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> users to manually workaround the problem.
>
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>

If the identified Linux change is correct, then the problems is almost
certainly the i915 driver not using appropriate cache flushing
operations on memory of weakened caching type.

Are there any details available on if/how/what snoop features the IGD
and its IOMMU have?

~Andrew

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[藍挺瑋]

Andrew Cooper 於 西元2015年07月18日 03:36 寫道:
> On 17/07/15 20:05, Ting-Wei Lan wrote:
>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>> devices, It is possible to encounter graphics issues that make screen
>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>
>> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>>
>> As we still cannot find a proper fix for this problem, this patch adds
>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>> users to manually workaround the problem.
>>
>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>
> If the identified Linux change is correct, then the problems is almost
> certainly the i915 driver not using appropriate cache flushing
> operations on memory of weakened caching type.
>
> Are there any details available on if/how/what snoop features the IGD
> and its IOMMU have?

How can I get these details? I get some messages from 'xl dmesg' or 
'lspci'. Are these messages related?

(XEN) PCI: Not using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d iommu 2 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation not enabled.
(XEN) Intel VT-d Interrupt Remapping not enabled.
(XEN) Intel VT-d Shared EPT tables not enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping disabled
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) Platform timer is 14.318MHz HPET
(XEN) Allocated console ring of 16 KiB.
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB

00:02.0 VGA compatible controller: Intel Corporation Core Processor 
Integrated Graphics Controller (rev 12) (prog-if 00 [VGA controller])
	Subsystem: ASUSTeK Computer Inc. Device 8383
	Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- 
Stepping- SERR- FastB2B- DisINTx+
	Status: Cap+ 66MHz- UDF- FastB2B+ ParErr- DEVSEL=fast >TAbort- <TAbort- 
<MAbort- >SERR- <PERR- INTx-
	Latency: 0
	Interrupt: pin A routed to IRQ 64
	Region 0: Memory at f7400000 (64-bit, non-prefetchable) [size=4M]
	Region 2: Memory at e0000000 (64-bit, prefetchable) [size=256M]
	Region 4: I/O ports at bc00 [size=8]
	Expansion ROM at <unassigned> [disabled]
	Capabilities: [90] MSI: Enable+ Count=1/1 Maskable- 64bit-
		Address: fee0400c  Data: 4191
	Capabilities: [d0] Power Management version 2
		Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [a4] PCI Advanced Features
		AFCap: TP+ FLR+
		AFCtrl: FLR-
		AFStatus: TP-
	Kernel driver in use: i915
	Kernel modules: i915

Can we get useful information from these Intel datasheets?
http://www.intel.com/content/www/us/en/processors/pentium/core-i5-600-i3-500-pentium-6000-datasheet-vol-1.html
http://www.intel.com/content/www/us/en/processors/pentium/core-i5-600-i3-500-pentium-6000-datasheet-vol-2.html

>
> ~Andrew
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Julien Grall]

Hi,

On 17/07/2015 20:05, Ting-Wei Lan wrote:
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
>
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> users to manually workaround the problem.

I would prefer to see the introduction of intel_iommu parameter in the 
vtd code rather than extending again the iommu parameter with intel 
specific options in the common code which is both for ARM and x86.

Regards,

-- 
Julien Grall

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Tian, Kevin]

> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> Sent: Saturday, July 18, 2015 3:06 AM
> 
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
> 
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> 
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> users to manually workaround the problem.
> 
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>

Since igfx works before, I'd think a more proper fix should be on the
bisected Linux commit or i915 to have two working correctly together.
Otherwise this patch is just hiding problem.

There is one possible usage to do selective IOMMU disable other than
global "iommu=off" switch. Then making this option general would
be better than igfx_off, e.g. based on BDF. But I'm not sure how it
is useful in reality.

> ---
>  docs/misc/xen-command-line.markdown  | 10 +++++++++-
>  xen/drivers/passthrough/iommu.c      |  3 +++
>  xen/drivers/passthrough/vtd/quirks.c |  2 +-
>  xen/include/xen/iommu.h              |  2 +-
>  4 files changed, 14 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.markdown
> b/docs/misc/xen-command-line.markdown
> index 13f03ad..7b61603 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -793,7 +793,7 @@ debug hypervisor only).
>  > Default: `new` unless directed-EOI is supported
> 
>  ### iommu
> -> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug
> | verbose | debug ]`
> +> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug
> | igfx_off | verbose | debug ]`
> 
>  > Sub-options:
> 
> @@ -867,6 +867,14 @@ debug hypervisor only).
>  >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
>  >> entry aren't PCI discoverable).
> 
> +> `igfx_off` (VT-d)
> +
> +> Default: `false`
> +
> +>> Workaround graphics issues for Intel Calpella/Ironlake devices. This option
> +>> is similar to Linux `intel_iommu=igfx_off`, so if it fixes anything, you
> +>> should file a bug reporting the problem.
> +
>  > `verbose`
> 
>  > Default: `false`
> diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
> index cc12735..2ba8c9d 100644
> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -47,6 +47,7 @@ bool_t __read_mostly force_iommu;
>  bool_t __hwdom_initdata iommu_dom0_strict;
>  bool_t __read_mostly iommu_verbose;
>  bool_t __read_mostly iommu_workaround_bios_bug;
> +bool_t __read_mostly iommu_igfx_off;
>  bool_t __read_mostly iommu_passthrough;
>  bool_t __read_mostly iommu_snoop = 1;
>  bool_t __read_mostly iommu_qinval = 1;
> @@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
>              force_iommu = val;
>          else if ( !strcmp(s, "workaround_bios_bug") )
>              iommu_workaround_bios_bug = val;
> +        else if ( !strcmp(s, "igfx_off") )
> +            iommu_igfx_off = val;
>          else if ( !strcmp(s, "verbose") )
>              iommu_verbose = val;
>          else if ( !strcmp(s, "snoop") )
> diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
> index 69d29ab..da1d853 100644
> --- a/xen/drivers/passthrough/vtd/quirks.c
> +++ b/xen/drivers/passthrough/vtd/quirks.c
> @@ -77,7 +77,7 @@ int is_igd_vt_enabled_quirk(void)
> 
>      /* integrated graphics on Intel platforms is located at 0:2.0 */
>      ggc = pci_conf_read16(0, 0, IGD_DEV, 0, GGC);
> -    return ( ggc & GGC_MEMORY_VT_ENABLED ? 1 : 0 );
> +    return ( ggc & GGC_MEMORY_VT_ENABLED ? 1 : 0 ) && !iommu_igfx_off;
>  }
> 
>  /*
> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> index 8eb764a..2cdb0a9 100644
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -29,7 +29,7 @@
> 
>  extern bool_t iommu_enable, iommu_enabled;
>  extern bool_t force_iommu, iommu_verbose;
> -extern bool_t iommu_workaround_bios_bug, iommu_passthrough;
> +extern bool_t iommu_workaround_bios_bug, iommu_igfx_off, iommu_passthrough;
>  extern bool_t iommu_snoop, iommu_qinval, iommu_intremap;
>  extern bool_t iommu_hap_pt_share;
>  extern bool_t iommu_debug;
> --
> 2.4.3

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Andrew Cooper]

On 20/07/2015 02:28, Tian, Kevin wrote:
>> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
>> Sent: Saturday, July 18, 2015 3:06 AM
>>
>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>> devices, It is possible to encounter graphics issues that make screen
>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>
>> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>>
>> As we still cannot find a proper fix for this problem, this patch adds
>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>> users to manually workaround the problem.
>>
>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> Since igfx works before, I'd think a more proper fix should be on the
> bisected Linux commit or i915 to have two working correctly together.
> Otherwise this patch is just hiding problem.

The linux commit is the one which actually fixes PAT support for Linux
under Xen.

It will cause the i915 driver to actually get WC mappings when it asks
for them.

> There is one possible usage to do selective IOMMU disable other than
> global "iommu=off" switch. Then making this option general would
> be better than igfx_off, e.g. based on BDF. But I'm not sure how it
> is useful in reality.

It is curious that just disabling the IOMMU appears to fix the problem. 
Are there any errata you are aware of on this class of system?

~Andrew

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Andrew Cooper]

On 19/07/2015 16:53, Julien Grall wrote:
> Hi,
>
> On 17/07/2015 20:05, Ting-Wei Lan wrote:
>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>> devices, It is possible to encounter graphics issues that make screen
>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>
>> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>>
>> As we still cannot find a proper fix for this problem, this patch adds
>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>> users to manually workaround the problem.
>
> I would prefer to see the introduction of intel_iommu parameter in the
> vtd code rather than extending again the iommu parameter with intel
> specific options in the common code which is both for ARM and x86.

Having the parsing for this option inside some x86 ifdefary, fine, but
lets not have multiple iommu= options.  It is hard enough for people to
drive one option (as people routinely get Xen and Linux command line
options mixed up).

~Andrew

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 17.07.15 at 21:05, <lantw44@gmail.com> wrote:
> @@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
>              force_iommu = val;
>          else if ( !strcmp(s, "workaround_bios_bug") )
>              iommu_workaround_bios_bug = val;
> +        else if ( !strcmp(s, "igfx_off") )
> +            iommu_igfx_off = val;

Just FTR (moot with both Kevin's and Andrew's replies, which I agree
with) this should be "igfx", with the intended use then being
"iommu=no-igfx".

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Julien Grall]

On 20/07/15 09:27, Andrew Cooper wrote:
> On 19/07/2015 16:53, Julien Grall wrote:
>> Hi,
>>
>> On 17/07/2015 20:05, Ting-Wei Lan wrote:
>>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>>> devices, It is possible to encounter graphics issues that make screen
>>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>>
>>> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>>>
>>> As we still cannot find a proper fix for this problem, this patch adds
>>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>>> users to manually workaround the problem.
>>
>> I would prefer to see the introduction of intel_iommu parameter in the
>> vtd code rather than extending again the iommu parameter with intel
>> specific options in the common code which is both for ARM and x86.
> 
> Having the parsing for this option inside some x86 ifdefary, fine, but
> lets not have multiple iommu= options.  It is hard enough for people to
> drive one option (as people routinely get Xen and Linux command line
> options mixed up).

If you are concerned about mixing Xen and Linux command line options, it
would be more sensible to use intel_iommu=igfx_off on Xen to keep the
options exactly the same as Linux.

Regards,

-- 
Julien Grall

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Ting-Wei Lan]

於 一，2015-07-20 於 09:21 +0100，Andrew Cooper 提到：
> On 20/07/2015 02:28, Tian, Kevin wrote:
> > > From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> > > Sent: Saturday, July 18, 2015 3:06 AM
> > > 
> > > When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel 
> > > Ironlake
> > > devices, It is possible to encounter graphics issues that make 
> > > screen
> > > unreadable or crash the system. It was reported in freedesktop 
> > > bugzilla:
> > > 
> > > https://bugs.freedesktop.org/show_bug.cgi?id=90037
> > > 
> > > As we still cannot find a proper fix for this problem, this patch 
> > > adds
> > > iommu=igfx_off option that is similar to Linux 
> > > intel_iommu=igfx_off for
> > > users to manually workaround the problem.
> > > 
> > > Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> > Since igfx works before, I'd think a more proper fix should be on 
> > the
> > bisected Linux commit or i915 to have two working correctly 
> > together.
> > Otherwise this patch is just hiding problem.
> 
> The linux commit is the one which actually fixes PAT support for 
> Linux
> under Xen.
> 
> It will cause the i915 driver to actually get WC mappings when it 
> asks
> for them.

I made a new i915 bug report because the old one seems to be ignored:
https://bugs.freedesktop.org/show_bug.cgi?id=91400

> 
> > There is one possible usage to do selective IOMMU disable other 
> > than
> > global "iommu=off" switch. Then making this option general would
> > be better than igfx_off, e.g. based on BDF. But I'm not sure how it
> > is useful in reality.
> 
> It is curious that just disabling the IOMMU appears to fix the 
> problem. 

Yes, disabling IOMMU (iommu=off or iommu=igfx_off added by this patch)
avoids the problem.

A similar problem also happens on Linux >= 3.7 without using Xen, and
it can also be avoided using intel_iommu=igfx_off.

> Are there any errata you are aware of on this class of system?

With Xen:
https://bugs.freedesktop.org/show_bug.cgi?id=90037
http://lists.xenproject.org/archives/html/xen-devel/2015
-06/msg02236.html
https://bugs.freedesktop.org/show_bug.cgi?id=91400

Without Xen:
https://bugs.freedesktop.org/show_bug.cgi?id=91127

> 
> ~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Andrew Cooper]

On 17/07/15 20:05, Ting-Wei Lan wrote:
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
>
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> users to manually workaround the problem.
>
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>

Having looked into this issue, the i915 driver has several workarounds
in it for systems when the IOMMU is in use.  In some cases there are
plain errata, while in other cases there are specific hardware features
which don't function if the IOMMU is enabled.

In all cases this is gated on Linux's idea of whether the IOMMU is
enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
or that Xen has turned it on.

Longterm, we should probably see about not clobbering the DMAR table and
making dom0 aware that there is an IOMMU.  (I still don't actually know
why we have this asymmetry between Intel and AMD).

However in the short term, having a command line workaround like this
seems appropriate.  It at least allows the user to tell Xen not to play
with the IOMMU behind an unsuspecting dom0.

~Andrew

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 20.07.15 at 14:12, <andrew.cooper3@citrix.com> wrote:
> On 17/07/15 20:05, Ting-Wei Lan wrote:
>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>> devices, It is possible to encounter graphics issues that make screen
>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>
>> https://bugs.freedesktop.org/show_bug.cgi?id=90037 
>>
>> As we still cannot find a proper fix for this problem, this patch adds
>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>> users to manually workaround the problem.
>>
>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> 
> Having looked into this issue, the i915 driver has several workarounds
> in it for systems when the IOMMU is in use.  In some cases there are
> plain errata, while in other cases there are specific hardware features
> which don't function if the IOMMU is enabled.
> 
> In all cases this is gated on Linux's idea of whether the IOMMU is
> enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
> or that Xen has turned it on.

Perhaps it should just assume an IOMMU is in use when running under
Xen. Having inspected all those code places quite some time ago, I
came to the conclusion that making this assumption is better than
the current one of there not being an enabled IOMMU (and I adjusted
our kernels accordingly).

> Longterm, we should probably see about not clobbering the DMAR table and
> making dom0 aware that there is an IOMMU.  (I still don't actually know
> why we have this asymmetry between Intel and AMD).

The presence of a DMAR table is not a 1:1 representation of whether
Xen has the IOMMU enabled.

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Andrew Cooper]

On 20/07/15 13:24, Jan Beulich wrote:
>>>> On 20.07.15 at 14:12, <andrew.cooper3@citrix.com> wrote:
>> On 17/07/15 20:05, Ting-Wei Lan wrote:
>>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>>> devices, It is possible to encounter graphics issues that make screen
>>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>>
>>> https://bugs.freedesktop.org/show_bug.cgi?id=90037 
>>>
>>> As we still cannot find a proper fix for this problem, this patch adds
>>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>>> users to manually workaround the problem.
>>>
>>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>> Having looked into this issue, the i915 driver has several workarounds
>> in it for systems when the IOMMU is in use.  In some cases there are
>> plain errata, while in other cases there are specific hardware features
>> which don't function if the IOMMU is enabled.
>>
>> In all cases this is gated on Linux's idea of whether the IOMMU is
>> enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
>> or that Xen has turned it on.
> Perhaps it should just assume an IOMMU is in use when running under
> Xen. Having inspected all those code places quite some time ago, I
> came to the conclusion that making this assumption is better than
> the current one of there not being an enabled IOMMU (and I adjusted
> our kernels accordingly).

In at least one case, an errata workaround involves issuing extra IOMMU
commands.  We cannot safely let even dom0 perform this.

>
>> Longterm, we should probably see about not clobbering the DMAR table and
>> making dom0 aware that there is an IOMMU.  (I still don't actually know
>> why we have this asymmetry between Intel and AMD).
> The presence of a DMAR table is not a 1:1 representation of whether
> Xen has the IOMMU enabled.

It would (in theory) allow dom0 to get RO mappings of the IOMMU control
pages and observe that the IOMMU is enabled.  Whether this is useful in
practice is a different matter.

~Andrew

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 20.07.15 at 14:34, <andrew.cooper3@citrix.com> wrote:
> On 20/07/15 13:24, Jan Beulich wrote:
>>>>> On 20.07.15 at 14:12, <andrew.cooper3@citrix.com> wrote:
>>> On 17/07/15 20:05, Ting-Wei Lan wrote:
>>>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>>>> devices, It is possible to encounter graphics issues that make screen
>>>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>>>
>>>> https://bugs.freedesktop.org/show_bug.cgi?id=90037 
>>>>
>>>> As we still cannot find a proper fix for this problem, this patch adds
>>>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>>>> users to manually workaround the problem.
>>>>
>>>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>>> Having looked into this issue, the i915 driver has several workarounds
>>> in it for systems when the IOMMU is in use.  In some cases there are
>>> plain errata, while in other cases there are specific hardware features
>>> which don't function if the IOMMU is enabled.
>>>
>>> In all cases this is gated on Linux's idea of whether the IOMMU is
>>> enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
>>> or that Xen has turned it on.
>> Perhaps it should just assume an IOMMU is in use when running under
>> Xen. Having inspected all those code places quite some time ago, I
>> came to the conclusion that making this assumption is better than
>> the current one of there not being an enabled IOMMU (and I adjusted
>> our kernels accordingly).
> 
> In at least one case, an errata workaround involves issuing extra IOMMU
> commands.  We cannot safely let even dom0 perform this.

Mind pointing out that one case? In our Xen kernels, IOMMU code
gets compiled out, so it is impossible for the driver to issue extra
IOMMU commands...

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Andrew Cooper]

On 20/07/15 14:55, Jan Beulich wrote:
>>>> On 20.07.15 at 14:34, <andrew.cooper3@citrix.com> wrote:
>> On 20/07/15 13:24, Jan Beulich wrote:
>>>>>> On 20.07.15 at 14:12, <andrew.cooper3@citrix.com> wrote:
>>>> On 17/07/15 20:05, Ting-Wei Lan wrote:
>>>>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>>>>> devices, It is possible to encounter graphics issues that make screen
>>>>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>>>>
>>>>> https://bugs.freedesktop.org/show_bug.cgi?id=90037 
>>>>>
>>>>> As we still cannot find a proper fix for this problem, this patch adds
>>>>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>>>>> users to manually workaround the problem.
>>>>>
>>>>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>>>> Having looked into this issue, the i915 driver has several workarounds
>>>> in it for systems when the IOMMU is in use.  In some cases there are
>>>> plain errata, while in other cases there are specific hardware features
>>>> which don't function if the IOMMU is enabled.
>>>>
>>>> In all cases this is gated on Linux's idea of whether the IOMMU is
>>>> enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
>>>> or that Xen has turned it on.
>>> Perhaps it should just assume an IOMMU is in use when running under
>>> Xen. Having inspected all those code places quite some time ago, I
>>> came to the conclusion that making this assumption is better than
>>> the current one of there not being an enabled IOMMU (and I adjusted
>>> our kernels accordingly).
>> In at least one case, an errata workaround involves issuing extra IOMMU
>> commands.  We cannot safely let even dom0 perform this.
> Mind pointing out that one case? In our Xen kernels, IOMMU code
> gets compiled out, so it is impossible for the driver to issue extra
> IOMMU commands...

But a distro is going to want to ship a single kernel, especially with
PVops these days.

~Andrew

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 20.07.15 at 16:12, <andrew.cooper3@citrix.com> wrote:
> On 20/07/15 14:55, Jan Beulich wrote:
>>>>> On 20.07.15 at 14:34, <andrew.cooper3@citrix.com> wrote:
>>> On 20/07/15 13:24, Jan Beulich wrote:
>>>>>>> On 20.07.15 at 14:12, <andrew.cooper3@citrix.com> wrote:
>>>>> On 17/07/15 20:05, Ting-Wei Lan wrote:
>>>>>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>>>>>> devices, It is possible to encounter graphics issues that make screen
>>>>>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>>>>>
>>>>>> https://bugs.freedesktop.org/show_bug.cgi?id=90037 
>>>>>>
>>>>>> As we still cannot find a proper fix for this problem, this patch adds
>>>>>> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>>>>>> users to manually workaround the problem.
>>>>>>
>>>>>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>>>>> Having looked into this issue, the i915 driver has several workarounds
>>>>> in it for systems when the IOMMU is in use.  In some cases there are
>>>>> plain errata, while in other cases there are specific hardware features
>>>>> which don't function if the IOMMU is enabled.
>>>>>
>>>>> In all cases this is gated on Linux's idea of whether the IOMMU is
>>>>> enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
>>>>> or that Xen has turned it on.
>>>> Perhaps it should just assume an IOMMU is in use when running under
>>>> Xen. Having inspected all those code places quite some time ago, I
>>>> came to the conclusion that making this assumption is better than
>>>> the current one of there not being an enabled IOMMU (and I adjusted
>>>> our kernels accordingly).
>>> In at least one case, an errata workaround involves issuing extra IOMMU
>>> commands.  We cannot safely let even dom0 perform this.
>> Mind pointing out that one case? In our Xen kernels, IOMMU code
>> gets compiled out, so it is impossible for the driver to issue extra
>> IOMMU commands...
> 
> But a distro is going to want to ship a single kernel, especially with
> PVops these days.

Right, but you see that there continue to be examples of what isn't
being taken care of in pv-ops.

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Tian, Kevin]

> From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew Cooper
> Sent: Monday, July 20, 2015 4:21 PM
> 
> On 20/07/2015 02:28, Tian, Kevin wrote:
> >> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> >> Sent: Saturday, July 18, 2015 3:06 AM
> >>
> >> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> >> devices, It is possible to encounter graphics issues that make screen
> >> unreadable or crash the system. It was reported in freedesktop bugzilla:
> >>
> >> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> >>
> >> As we still cannot find a proper fix for this problem, this patch adds
> >> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> >> users to manually workaround the problem.
> >>
> >> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> > Since igfx works before, I'd think a more proper fix should be on the
> > bisected Linux commit or i915 to have two working correctly together.
> > Otherwise this patch is just hiding problem.
> 
> The linux commit is the one which actually fixes PAT support for Linux
> under Xen.
> 
> It will cause the i915 driver to actually get WC mappings when it asks
> for them.

This is the part which I don't quite understand. WC is essentially an UC
attribute with write buffer to accelerate the write efficiency. There 
should be no correctness problem to use either WC or UC if i915 driver
wants WC.

> 
> > There is one possible usage to do selective IOMMU disable other than
> > global "iommu=off" switch. Then making this option general would
> > be better than igfx_off, e.g. based on BDF. But I'm not sure how it
> > is useful in reality.
> 
> It is curious that just disabling the IOMMU appears to fix the problem.
> Are there any errata you are aware of on this class of system?
> 

No such errata in my mind. It's not caused by IOMMU HW, since 
dom0-passthrough can't solve the problem neither. It's possible our 
pvMMU code may explicitly check presence of IOMMU to behave 
differently on PAT setting, but I'm not familiar with that part.

Thanks
Kevin

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Tian, Kevin]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Monday, July 20, 2015 8:24 PM
> 
> >>> On 20.07.15 at 14:12, <andrew.cooper3@citrix.com> wrote:
> > On 17/07/15 20:05, Ting-Wei Lan wrote:
> >> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> >> devices, It is possible to encounter graphics issues that make screen
> >> unreadable or crash the system. It was reported in freedesktop bugzilla:
> >>
> >> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> >>
> >> As we still cannot find a proper fix for this problem, this patch adds
> >> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> >> users to manually workaround the problem.
> >>
> >> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> >
> > Having looked into this issue, the i915 driver has several workarounds
> > in it for systems when the IOMMU is in use.  In some cases there are
> > plain errata, while in other cases there are specific hardware features
> > which don't function if the IOMMU is enabled.
> >
> > In all cases this is gated on Linux's idea of whether the IOMMU is
> > enabled.  When used under Xen, Linux has no clue that the IOMMU exists,
> > or that Xen has turned it on.
> 
> Perhaps it should just assume an IOMMU is in use when running under
> Xen. Having inspected all those code places quite some time ago, I
> came to the conclusion that making this assumption is better than
> the current one of there not being an enabled IOMMU (and I adjusted
> our kernels accordingly).

kind of agree here. IIRC, to have i915 working correctly in Dom0, user
needs to manually turn on CONFIG_DMAR and CONFIG_INTEL_IOMMU
although there's no IOMMU exposed. Otherwise i915 driver will use plain 
virt_to_phys when programming GTT table which causes trouble. 
Recently there are some improvements in this part, but still some IOMMU
specific tricks remain. So having the options default on under Xen looks
better.

Thanks
Kevin

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 21.07.15 at 02:57, <kevin.tian@intel.com> wrote:
>>  From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew 
> Cooper
>> Sent: Monday, July 20, 2015 4:21 PM
>> 
>> On 20/07/2015 02:28, Tian, Kevin wrote:
>> >> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
>> >> Sent: Saturday, July 18, 2015 3:06 AM
>> >>
>> >> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>> >> devices, It is possible to encounter graphics issues that make screen
>> >> unreadable or crash the system. It was reported in freedesktop bugzilla:
>> >>
>> >> https://bugs.freedesktop.org/show_bug.cgi?id=90037 
>> >>
>> >> As we still cannot find a proper fix for this problem, this patch adds
>> >> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
>> >> users to manually workaround the problem.
>> >>
>> >> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>> > Since igfx works before, I'd think a more proper fix should be on the
>> > bisected Linux commit or i915 to have two working correctly together.
>> > Otherwise this patch is just hiding problem.
>> 
>> The linux commit is the one which actually fixes PAT support for Linux
>> under Xen.
>> 
>> It will cause the i915 driver to actually get WC mappings when it asks
>> for them.
> 
> This is the part which I don't quite understand. WC is essentially an UC
> attribute with write buffer to accelerate the write efficiency. There 
> should be no correctness problem to use either WC or UC if i915 driver
> wants WC.

"Should" is too weak a term here: Using WC on the wrong piece of
memory or without the necessary fencing can imo very well cause
correctness problems (which would be hidden by WC -> UC
conversion behind the driver's back).

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Tian, Kevin]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Tuesday, July 21, 2015 2:57 PM
> 
> >>> On 21.07.15 at 02:57, <kevin.tian@intel.com> wrote:
> >>  From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew
> > Cooper
> >> Sent: Monday, July 20, 2015 4:21 PM
> >>
> >> On 20/07/2015 02:28, Tian, Kevin wrote:
> >> >> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> >> >> Sent: Saturday, July 18, 2015 3:06 AM
> >> >>
> >> >> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> >> >> devices, It is possible to encounter graphics issues that make screen
> >> >> unreadable or crash the system. It was reported in freedesktop bugzilla:
> >> >>
> >> >> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> >> >>
> >> >> As we still cannot find a proper fix for this problem, this patch adds
> >> >> iommu=igfx_off option that is similar to Linux intel_iommu=igfx_off for
> >> >> users to manually workaround the problem.
> >> >>
> >> >> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> >> > Since igfx works before, I'd think a more proper fix should be on the
> >> > bisected Linux commit or i915 to have two working correctly together.
> >> > Otherwise this patch is just hiding problem.
> >>
> >> The linux commit is the one which actually fixes PAT support for Linux
> >> under Xen.
> >>
> >> It will cause the i915 driver to actually get WC mappings when it asks
> >> for them.
> >
> > This is the part which I don't quite understand. WC is essentially an UC
> > attribute with write buffer to accelerate the write efficiency. There
> > should be no correctness problem to use either WC or UC if i915 driver
> > wants WC.
> 
> "Should" is too weak a term here: Using WC on the wrong piece of
> memory or without the necessary fencing can imo very well cause
> correctness problems (which would be hidden by WC -> UC
> conversion behind the driver's back).
> 

My point is about when i915 wants WC, then either UC (I suppose is
the case before that Linux commit) and WC (by that commit) has
no correctness problem. UC is more strict than WC. It's just performance 
difference. It's not about using WC in wrong place when it's not desired.

Thanks
Kevin

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 21.07.15 at 09:05, <kevin.tian@intel.com> wrote:
>>  From: Jan Beulich [mailto:JBeulich@suse.com]
>> Sent: Tuesday, July 21, 2015 2:57 PM
>> >>> On 21.07.15 at 02:57, <kevin.tian@intel.com> wrote:
>> >>  From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew
>> > Cooper
>> >> Sent: Monday, July 20, 2015 4:21 PM
>> > This is the part which I don't quite understand. WC is essentially an UC
>> > attribute with write buffer to accelerate the write efficiency. There
>> > should be no correctness problem to use either WC or UC if i915 driver
>> > wants WC.
>> 
>> "Should" is too weak a term here: Using WC on the wrong piece of
>> memory or without the necessary fencing can imo very well cause
>> correctness problems (which would be hidden by WC -> UC
>> conversion behind the driver's back).
>> 
> 
> My point is about when i915 wants WC, then either UC (I suppose is
> the case before that Linux commit) and WC (by that commit) has
> no correctness problem. UC is more strict than WC. It's just performance 
> difference. It's not about using WC in wrong place when it's not desired.

In this you assume there are no misguided attempts to request
WC in the driver, which would have gone unnoticed as long as WC
didn't become the effective attribute. And "misguided" here is
meant to include cases where hardware errata may need taking
care of.

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Tian, Kevin]

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Tuesday, July 21, 2015 3:17 PM
 
> >>> On 21.07.15 at 09:05, <kevin.tian@intel.com> wrote:
> >>  From: Jan Beulich [mailto:JBeulich@suse.com]
> >> Sent: Tuesday, July 21, 2015 2:57 PM
> >> >>> On 21.07.15 at 02:57, <kevin.tian@intel.com> wrote:
> >> >>  From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew
> >> > Cooper
> >> >> Sent: Monday, July 20, 2015 4:21 PM
> >> > This is the part which I don't quite understand. WC is essentially an UC
> >> > attribute with write buffer to accelerate the write efficiency. There
> >> > should be no correctness problem to use either WC or UC if i915 driver
> >> > wants WC.
> >>
> >> "Should" is too weak a term here: Using WC on the wrong piece of
> >> memory or without the necessary fencing can imo very well cause
> >> correctness problems (which would be hidden by WC -> UC
> >> conversion behind the driver's back).
> >>
> >
> > My point is about when i915 wants WC, then either UC (I suppose is
> > the case before that Linux commit) and WC (by that commit) has
> > no correctness problem. UC is more strict than WC. It's just performance
> > difference. It's not about using WC in wrong place when it's not desired.
> 
> In this you assume there are no misguided attempts to request
> WC in the driver, which would have gone unnoticed as long as WC
> didn't become the effective attribute. And "misguided" here is
> meant to include cases where hardware errata may need taking
> care of.
> 

I don't understand this point. If it's misguided attempts it'd be
same on bare metal. 

Thanks
Kevin

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Jan Beulich]

>>> On 21.07.15 at 09:23, <kevin.tian@intel.com> wrote:
>>  From: Jan Beulich [mailto:JBeulich@suse.com]
>> Sent: Tuesday, July 21, 2015 3:17 PM
>  
>> >>> On 21.07.15 at 09:05, <kevin.tian@intel.com> wrote:
>> >>  From: Jan Beulich [mailto:JBeulich@suse.com]
>> >> Sent: Tuesday, July 21, 2015 2:57 PM
>> >> >>> On 21.07.15 at 02:57, <kevin.tian@intel.com> wrote:
>> >> >>  From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew
>> >> > Cooper
>> >> >> Sent: Monday, July 20, 2015 4:21 PM
>> >> > This is the part which I don't quite understand. WC is essentially an UC
>> >> > attribute with write buffer to accelerate the write efficiency. There
>> >> > should be no correctness problem to use either WC or UC if i915 driver
>> >> > wants WC.
>> >>
>> >> "Should" is too weak a term here: Using WC on the wrong piece of
>> >> memory or without the necessary fencing can imo very well cause
>> >> correctness problems (which would be hidden by WC -> UC
>> >> conversion behind the driver's back).
>> >>
>> >
>> > My point is about when i915 wants WC, then either UC (I suppose is
>> > the case before that Linux commit) and WC (by that commit) has
>> > no correctness problem. UC is more strict than WC. It's just performance
>> > difference. It's not about using WC in wrong place when it's not desired.
>> 
>> In this you assume there are no misguided attempts to request
>> WC in the driver, which would have gone unnoticed as long as WC
>> didn't become the effective attribute. And "misguided" here is
>> meant to include cases where hardware errata may need taking
>> care of.
> 
> I don't understand this point. If it's misguided attempts it'd be
> same on bare metal. 

Not if bare metal has a workaround in place depending on (or even
implemented in) IOMMU code. Also iirc the reported said that a
similar problem existed (exists?) on native too.

Jan

Re: [PATCH] VT-d: add iommu=igfx_off option to workaround graphics issues

[Ting-Wei Lan]

Jan Beulich 於 西元2015年07月21日 15:33 寫道:
>>>> On 21.07.15 at 09:23, <kevin.tian@intel.com> wrote:
>>>   From: Jan Beulich [mailto:JBeulich@suse.com]
>>> Sent: Tuesday, July 21, 2015 3:17 PM
>>
>>>>>> On 21.07.15 at 09:05, <kevin.tian@intel.com> wrote:
>>>>>   From: Jan Beulich [mailto:JBeulich@suse.com]
>>>>> Sent: Tuesday, July 21, 2015 2:57 PM
>>>>>>>> On 21.07.15 at 02:57, <kevin.tian@intel.com> wrote:
>>>>>>>   From: Andrew Cooper [mailto:amc96@hermes.cam.ac.uk] On Behalf Of Andrew
>>>>>> Cooper
>>>>>>> Sent: Monday, July 20, 2015 4:21 PM
>>>>>> This is the part which I don't quite understand. WC is essentially an UC
>>>>>> attribute with write buffer to accelerate the write efficiency. There
>>>>>> should be no correctness problem to use either WC or UC if i915 driver
>>>>>> wants WC.
>>>>>
>>>>> "Should" is too weak a term here: Using WC on the wrong piece of
>>>>> memory or without the necessary fencing can imo very well cause
>>>>> correctness problems (which would be hidden by WC -> UC
>>>>> conversion behind the driver's back).
>>>>>
>>>>
>>>> My point is about when i915 wants WC, then either UC (I suppose is
>>>> the case before that Linux commit) and WC (by that commit) has
>>>> no correctness problem. UC is more strict than WC. It's just performance
>>>> difference. It's not about using WC in wrong place when it's not desired.
>>>
>>> In this you assume there are no misguided attempts to request
>>> WC in the driver, which would have gone unnoticed as long as WC
>>> didn't become the effective attribute. And "misguided" here is
>>> meant to include cases where hardware errata may need taking
>>> care of.
>>
>> I don't understand this point. If it's misguided attempts it'd be
>> same on bare metal.
>
> Not if bare metal has a workaround in place depending on (or even
> implemented in) IOMMU code. Also iirc the reported said that a
> similar problem existed (exists?) on native too.

On 3.7 <= Linux < 4.2-rc2 without Xen, the screen output is broken, and 
the system crashes after display server is started.

On Linux >= 4.2-rc2 without Xen, the screen output is normal because 
this patch is added
https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=8b572a4
but these messages are showed every a few seconds. If I keep using it, 
it will crash in about two hours.
DMAR: DMAR:[DMA Write] Request device [00:02.0] fault addr fdf70000
DMAR:[fault reason 05] PTE Write access is not set

On Linux >= 3.19 with Xen, the screen output is broken, and these 
similar messages are showed.
(XEN) [VT-D]DMAR:[DMA Write] Request device [0000:00:02.0] fault addr 
73fbff000, iommu reg = ffff82c000203000
(XEN) [VT-D]DMAR: reason 05 - PTE Write access is not set

>
> Jan
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v2] VT-d: add iommu=igfx option to workaround graphics issues

[Ting-Wei Lan]

When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
devices, It is possible to encounter graphics issues that make screen
unreadable or crash the system. It was reported in freedesktop bugzilla:

https://bugs.freedesktop.org/show_bug.cgi?id=90037

As we still cannot find a proper fix for this problem, this patch adds
iommu=igfx option to control whether Intel graphics IOMMU is enabled on
these devices. Running Xen with iommu=no-igfx is similar to running
Linux with intel_iommu=igfx_off, which disables IOMMU for Intel Ironlake
GPU. This can be used by users to manually workaround the problem before
a fix is available for i915 driver.

Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
---
Changed since v1:
  * Replace igfx_off with igfx

This patch is currently only build-tested because I don't have access to
the hardware that have graphics issues this week. I will test this new
patch next week.

 docs/misc/xen-command-line.markdown  | 12 +++++++++++-
 xen/drivers/passthrough/iommu.c      |  3 +++
 xen/drivers/passthrough/vtd/quirks.c |  2 +-
 xen/include/xen/iommu.h              |  2 +-
 4 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 13f03ad..6262be6 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -793,7 +793,7 @@ debug hypervisor only).
 > Default: `new` unless directed-EOI is supported
 
 ### iommu
-> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | verbose | debug ]`
+> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | igfx | verbose | debug ]`
 
 > Sub-options:
 
@@ -867,6 +867,16 @@ debug hypervisor only).
 >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
 >> entry aren't PCI discoverable).
 
+> `igfx` (VT-d)
+
+> Default: `true`
+
+>> Enable IOMMU for Intel Calpella/Ironlake devices. This option does not
+>> affect grahpics IOMMU on other devices. The intended usage of this option
+>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
+>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
+>> should file a bug reporting the problem.
+
 > `verbose`
 
 > Default: `false`
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index cc12735..966cc66 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -47,6 +47,7 @@ bool_t __read_mostly force_iommu;
 bool_t __hwdom_initdata iommu_dom0_strict;
 bool_t __read_mostly iommu_verbose;
 bool_t __read_mostly iommu_workaround_bios_bug;
+bool_t __read_mostly iommu_igfx = 1;
 bool_t __read_mostly iommu_passthrough;
 bool_t __read_mostly iommu_snoop = 1;
 bool_t __read_mostly iommu_qinval = 1;
@@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
             force_iommu = val;
         else if ( !strcmp(s, "workaround_bios_bug") )
             iommu_workaround_bios_bug = val;
+        else if ( !strcmp(s, "igfx") )
+            iommu_igfx = val;
         else if ( !strcmp(s, "verbose") )
             iommu_verbose = val;
         else if ( !strcmp(s, "snoop") )
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 69d29ab..2eac1cf 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -77,7 +77,7 @@ int is_igd_vt_enabled_quirk(void)
 
     /* integrated graphics on Intel platforms is located at 0:2.0 */
     ggc = pci_conf_read16(0, 0, IGD_DEV, 0, GGC);
-    return ( ggc & GGC_MEMORY_VT_ENABLED ? 1 : 0 );
+    return ( ggc & GGC_MEMORY_VT_ENABLED ? 1 : 0 ) && iommu_igfx;
 }
 
 /*
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8eb764a..29eed51 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -29,7 +29,7 @@
 
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
-extern bool_t iommu_workaround_bios_bug, iommu_passthrough;
+extern bool_t iommu_workaround_bios_bug, iommu_igfx, iommu_passthrough;
 extern bool_t iommu_snoop, iommu_qinval, iommu_intremap;
 extern bool_t iommu_hap_pt_share;
 extern bool_t iommu_debug;
-- 
2.4.3

Re: [PATCH v2] VT-d: add iommu=igfx option to workaround graphics issues

[Andrew Cooper]

On 25/07/2015 17:57, Ting-Wei Lan wrote:
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
>
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx option to control whether Intel graphics IOMMU is enabled on
> these devices. Running Xen with iommu=no-igfx is similar to running
> Linux with intel_iommu=igfx_off, which disables IOMMU for Intel Ironlake
> GPU. This can be used by users to manually workaround the problem before
> a fix is available for i915 driver.
>
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>

Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Re: [PATCH v2] VT-d: add iommu=igfx option to workaround graphics issues

[Tian, Kevin]

> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> Sent: Sunday, July 26, 2015 12:58 AM
> 
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
> 
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> 
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx option to control whether Intel graphics IOMMU is enabled on
> these devices. Running Xen with iommu=no-igfx is similar to running
> Linux with intel_iommu=igfx_off, which disables IOMMU for Intel Ironlake
> GPU. This can be used by users to manually workaround the problem before
> a fix is available for i915 driver.
> 
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> ---
> Changed since v1:
>   * Replace igfx_off with igfx
> 
> This patch is currently only build-tested because I don't have access to
> the hardware that have graphics issues this week. I will test this new
> patch next week.
> 
>  docs/misc/xen-command-line.markdown  | 12 +++++++++++-
>  xen/drivers/passthrough/iommu.c      |  3 +++
>  xen/drivers/passthrough/vtd/quirks.c |  2 +-
>  xen/include/xen/iommu.h              |  2 +-
>  4 files changed, 16 insertions(+), 3 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.markdown
> b/docs/misc/xen-command-line.markdown
> index 13f03ad..6262be6 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -793,7 +793,7 @@ debug hypervisor only).
>  > Default: `new` unless directed-EOI is supported
> 
>  ### iommu
> -> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug
> | verbose | debug ]`
> +> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug
> | igfx | verbose | debug ]`
> 
>  > Sub-options:
> 
> @@ -867,6 +867,16 @@ debug hypervisor only).
>  >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
>  >> entry aren't PCI discoverable).
> 
> +> `igfx` (VT-d)
> +
> +> Default: `true`
> +
> +>> Enable IOMMU for Intel Calpella/Ironlake devices. This option does not
> +>> affect grahpics IOMMU on other devices. The intended usage of this option
> +>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
> +>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
> +>> should file a bug reporting the problem.
> +

For above description let's make it general, i.e Enable IOMMU for Intel Processor
Graphics devices. You can list Calpella/Ironlake as the example, but not the only
target. :-)

Thanks
Kevin

Re: [PATCH v2] VT-d: add iommu=igfx option to workaround graphics issues

[Ting-Wei Lan]

Tian, Kevin 於 西元2015年07月31日 09:26 寫道:
>> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
>> Sent: Sunday, July 26, 2015 12:58 AM
>>
>> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
>> devices, It is possible to encounter graphics issues that make screen
>> unreadable or crash the system. It was reported in freedesktop bugzilla:
>>
>> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>>
>> As we still cannot find a proper fix for this problem, this patch adds
>> iommu=igfx option to control whether Intel graphics IOMMU is enabled on
>> these devices. Running Xen with iommu=no-igfx is similar to running
>> Linux with intel_iommu=igfx_off, which disables IOMMU for Intel Ironlake
>> GPU. This can be used by users to manually workaround the problem before
>> a fix is available for i915 driver.
>>
>> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
>> ---
>> Changed since v1:
>>    * Replace igfx_off with igfx
>>
>> This patch is currently only build-tested because I don't have access to
>> the hardware that have graphics issues this week. I will test this new
>> patch next week.
>>
>>   docs/misc/xen-command-line.markdown  | 12 +++++++++++-
>>   xen/drivers/passthrough/iommu.c      |  3 +++
>>   xen/drivers/passthrough/vtd/quirks.c |  2 +-
>>   xen/include/xen/iommu.h              |  2 +-
>>   4 files changed, 16 insertions(+), 3 deletions(-)
>>
>> diff --git a/docs/misc/xen-command-line.markdown
>> b/docs/misc/xen-command-line.markdown
>> index 13f03ad..6262be6 100644
>> --- a/docs/misc/xen-command-line.markdown
>> +++ b/docs/misc/xen-command-line.markdown
>> @@ -793,7 +793,7 @@ debug hypervisor only).
>>   > Default: `new` unless directed-EOI is supported
>>
>>   ### iommu
>> -> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
>> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug
>> | verbose | debug ]`
>> +> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
>> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug
>> | igfx | verbose | debug ]`
>>
>>   > Sub-options:
>>
>> @@ -867,6 +867,16 @@ debug hypervisor only).
>>   >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
>>   >> entry aren't PCI discoverable).
>>
>> +> `igfx` (VT-d)
>> +
>> +> Default: `true`
>> +
>> +>> Enable IOMMU for Intel Calpella/Ironlake devices. This option does not
>> +>> affect grahpics IOMMU on other devices. The intended usage of this option
>> +>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
>> +>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
>> +>> should file a bug reporting the problem.
>> +
>
> For above description let's make it general, i.e Enable IOMMU for Intel Processor
> Graphics devices. You can list Calpella/Ironlake as the example, but not the only
> target. :-)

no-igfx only works with Calpella/Ironlake because 
is_igd_vt_enabled_quirk() contains this code:

if ( !IS_ILK(ioh_id) )
     return 1;

The newly added variable iommu_igfx only changes the return value of 
is_igd_vt_enabled_quirk() when IS_ILK(ioh_id) is true. I don't know 
whether we will need no-igfx on other Intel graphics devices because I 
don't many devices to test and I don't have other devices that have this 
problem.

>
> Thanks
> Kevin
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [PATCH v2] VT-d: add iommu=igfx option to workaround graphics issues

[Tian, Kevin]

> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> Sent: Friday, July 31, 2015 4:38 PM
> 
> Tian, Kevin 於 西元2015年07月31日 09:26 寫道:
> >> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> >> Sent: Sunday, July 26, 2015 12:58 AM
> >>
> >> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> >> devices, It is possible to encounter graphics issues that make screen
> >> unreadable or crash the system. It was reported in freedesktop bugzilla:
> >>
> >> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> >>
> >> As we still cannot find a proper fix for this problem, this patch adds
> >> iommu=igfx option to control whether Intel graphics IOMMU is enabled on
> >> these devices. Running Xen with iommu=no-igfx is similar to running
> >> Linux with intel_iommu=igfx_off, which disables IOMMU for Intel Ironlake
> >> GPU. This can be used by users to manually workaround the problem before
> >> a fix is available for i915 driver.
> >>
> >> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> >> ---
> >> Changed since v1:
> >>    * Replace igfx_off with igfx
> >>
> >> This patch is currently only build-tested because I don't have access to
> >> the hardware that have graphics issues this week. I will test this new
> >> patch next week.
> >>
> >>   docs/misc/xen-command-line.markdown  | 12 +++++++++++-
> >>   xen/drivers/passthrough/iommu.c      |  3 +++
> >>   xen/drivers/passthrough/vtd/quirks.c |  2 +-
> >>   xen/include/xen/iommu.h              |  2 +-
> >>   4 files changed, 16 insertions(+), 3 deletions(-)
> >>
> >> diff --git a/docs/misc/xen-command-line.markdown
> >> b/docs/misc/xen-command-line.markdown
> >> index 13f03ad..6262be6 100644
> >> --- a/docs/misc/xen-command-line.markdown
> >> +++ b/docs/misc/xen-command-line.markdown
> >> @@ -793,7 +793,7 @@ debug hypervisor only).
> >>   > Default: `new` unless directed-EOI is supported
> >>
> >>   ### iommu
> >> -> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
> >> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap |
> workaround_bios_bug
> >> | verbose | debug ]`
> >> +> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept |
> >> dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap |
> workaround_bios_bug
> >> | igfx | verbose | debug ]`
> >>
> >>   > Sub-options:
> >>
> >> @@ -867,6 +867,16 @@ debug hypervisor only).
> >>   >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
> >>   >> entry aren't PCI discoverable).
> >>
> >> +> `igfx` (VT-d)
> >> +
> >> +> Default: `true`
> >> +
> >> +>> Enable IOMMU for Intel Calpella/Ironlake devices. This option does not
> >> +>> affect grahpics IOMMU on other devices. The intended usage of this option
> >> +>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
> >> +>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
> >> +>> should file a bug reporting the problem.
> >> +
> >
> > For above description let's make it general, i.e Enable IOMMU for Intel Processor
> > Graphics devices. You can list Calpella/Ironlake as the example, but not the only
> > target. :-)
> 
> no-igfx only works with Calpella/Ironlake because
> is_igd_vt_enabled_quirk() contains this code:
> 
> if ( !IS_ILK(ioh_id) )
>      return 1;
> 

Then let's move the no-igfx check out of this function. 

> The newly added variable iommu_igfx only changes the return value of
> is_igd_vt_enabled_quirk() when IS_ILK(ioh_id) is true. I don't know
> whether we will need no-igfx on other Intel graphics devices because I
> don't many devices to test and I don't have other devices that have this
> problem.
> 

The option itself says nothing Calpella/Ironlake specific, and an user will
intercept it as a general option for all igfx devices so it'd be strange to
see this option no effect on other platform.

Thanks
Kevin
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

[PATCH v3] VT-d: add iommu=igfx option to workaround graphics issues

[Ting-Wei Lan]

When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
devices, It is possible to encounter graphics issues that make screen
unreadable or crash the system. It was reported in freedesktop bugzilla:

https://bugs.freedesktop.org/show_bug.cgi?id=90037

As we still cannot find a proper fix for this problem, this patch adds
iommu=igfx option to control whether Intel graphics IOMMU is enabled.
Running Xen with iommu=no-igfx is similar to running Linux with
intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
used by users to manually workaround the problem before a fix is
available for i915 driver.

Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
---
Changed since v2:
  * Make no-igfx available for all Intel devices, not just Calpella/Ironlake
Changed since v1:
  * Replace igfx_off with igfx

 docs/misc/xen-command-line.markdown  | 11 ++++++++++-
 xen/drivers/passthrough/iommu.c      |  3 +++
 xen/drivers/passthrough/vtd/quirks.c |  3 +++
 xen/include/xen/iommu.h              |  2 +-
 4 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 13f03ad..486e53b 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -793,7 +793,7 @@ debug hypervisor only).
 > Default: `new` unless directed-EOI is supported
 
 ### iommu
-> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | verbose | debug ]`
+> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | igfx | verbose | debug ]`
 
 > Sub-options:
 
@@ -867,6 +867,15 @@ debug hypervisor only).
 >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
 >> entry aren't PCI discoverable).
 
+> `igfx` (VT-d)
+
+> Default: `true`
+
+>> Enable IOMMU for Intel graphics devices. The intended usage of this option
+>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
+>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
+>> should file a bug reporting the problem.
+
 > `verbose`
 
 > Default: `false`
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index cc12735..966cc66 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -47,6 +47,7 @@ bool_t __read_mostly force_iommu;
 bool_t __hwdom_initdata iommu_dom0_strict;
 bool_t __read_mostly iommu_verbose;
 bool_t __read_mostly iommu_workaround_bios_bug;
+bool_t __read_mostly iommu_igfx = 1;
 bool_t __read_mostly iommu_passthrough;
 bool_t __read_mostly iommu_snoop = 1;
 bool_t __read_mostly iommu_qinval = 1;
@@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
             force_iommu = val;
         else if ( !strcmp(s, "workaround_bios_bug") )
             iommu_workaround_bios_bug = val;
+        else if ( !strcmp(s, "igfx") )
+            iommu_igfx = val;
         else if ( !strcmp(s, "verbose") )
             iommu_verbose = val;
         else if ( !strcmp(s, "snoop") )
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 69d29ab..92cff1d 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -72,6 +72,9 @@ int is_igd_vt_enabled_quirk(void)
 {
     u16 ggc;
 
+    if ( !iommu_igfx )
+        return 0;
+
     if ( !IS_ILK(ioh_id) )
         return 1;
 
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8eb764a..29eed51 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -29,7 +29,7 @@
 
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
-extern bool_t iommu_workaround_bios_bug, iommu_passthrough;
+extern bool_t iommu_workaround_bios_bug, iommu_igfx, iommu_passthrough;
 extern bool_t iommu_snoop, iommu_qinval, iommu_intremap;
 extern bool_t iommu_hap_pt_share;
 extern bool_t iommu_debug;
-- 
2.4.3

Re: [PATCH v3] VT-d: add iommu=igfx option to workaround graphics issues

[Andrew Cooper]

On 05/08/15 10:11, Ting-Wei Lan wrote:
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
>
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
>
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx option to control whether Intel graphics IOMMU is enabled.
> Running Xen with iommu=no-igfx is similar to running Linux with
> intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
> used by users to manually workaround the problem before a fix is
> available for i915 driver.
>
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>

Wei: I think this is a candidate for inclusion into 4.6

~Andrew

> ---
> Changed since v2:
>   * Make no-igfx available for all Intel devices, not just Calpella/Ironlake
> Changed since v1:
>   * Replace igfx_off with igfx
>
>  docs/misc/xen-command-line.markdown  | 11 ++++++++++-
>  xen/drivers/passthrough/iommu.c      |  3 +++
>  xen/drivers/passthrough/vtd/quirks.c |  3 +++
>  xen/include/xen/iommu.h              |  2 +-
>  4 files changed, 17 insertions(+), 2 deletions(-)
>
> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
> index 13f03ad..486e53b 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -793,7 +793,7 @@ debug hypervisor only).
>  > Default: `new` unless directed-EOI is supported
>  
>  ### iommu
> -> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | verbose | debug ]`
> +> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | igfx | verbose | debug ]`
>  
>  > Sub-options:
>  
> @@ -867,6 +867,15 @@ debug hypervisor only).
>  >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
>  >> entry aren't PCI discoverable).
>  
> +> `igfx` (VT-d)
> +
> +> Default: `true`
> +
> +>> Enable IOMMU for Intel graphics devices. The intended usage of this option
> +>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
> +>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
> +>> should file a bug reporting the problem.
> +
>  > `verbose`
>  
>  > Default: `false`
> diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
> index cc12735..966cc66 100644
> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -47,6 +47,7 @@ bool_t __read_mostly force_iommu;
>  bool_t __hwdom_initdata iommu_dom0_strict;
>  bool_t __read_mostly iommu_verbose;
>  bool_t __read_mostly iommu_workaround_bios_bug;
> +bool_t __read_mostly iommu_igfx = 1;
>  bool_t __read_mostly iommu_passthrough;
>  bool_t __read_mostly iommu_snoop = 1;
>  bool_t __read_mostly iommu_qinval = 1;
> @@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
>              force_iommu = val;
>          else if ( !strcmp(s, "workaround_bios_bug") )
>              iommu_workaround_bios_bug = val;
> +        else if ( !strcmp(s, "igfx") )
> +            iommu_igfx = val;
>          else if ( !strcmp(s, "verbose") )
>              iommu_verbose = val;
>          else if ( !strcmp(s, "snoop") )
> diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
> index 69d29ab..92cff1d 100644
> --- a/xen/drivers/passthrough/vtd/quirks.c
> +++ b/xen/drivers/passthrough/vtd/quirks.c
> @@ -72,6 +72,9 @@ int is_igd_vt_enabled_quirk(void)
>  {
>      u16 ggc;
>  
> +    if ( !iommu_igfx )
> +        return 0;
> +
>      if ( !IS_ILK(ioh_id) )
>          return 1;
>  
> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> index 8eb764a..29eed51 100644
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -29,7 +29,7 @@
>  
>  extern bool_t iommu_enable, iommu_enabled;
>  extern bool_t force_iommu, iommu_verbose;
> -extern bool_t iommu_workaround_bios_bug, iommu_passthrough;
> +extern bool_t iommu_workaround_bios_bug, iommu_igfx, iommu_passthrough;
>  extern bool_t iommu_snoop, iommu_qinval, iommu_intremap;
>  extern bool_t iommu_hap_pt_share;
>  extern bool_t iommu_debug;

Re: [PATCH v3] VT-d: add iommu=igfx option to workaround graphics issues

[Wei Liu]

On Wed, Aug 05, 2015 at 01:18:05PM +0100, Andrew Cooper wrote:
> On 05/08/15 10:11, Ting-Wei Lan wrote:
> > When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> > devices, It is possible to encounter graphics issues that make screen
> > unreadable or crash the system. It was reported in freedesktop bugzilla:
> >
> > https://bugs.freedesktop.org/show_bug.cgi?id=90037
> >
> > As we still cannot find a proper fix for this problem, this patch adds
> > iommu=igfx option to control whether Intel graphics IOMMU is enabled.
> > Running Xen with iommu=no-igfx is similar to running Linux with
> > intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
> > used by users to manually workaround the problem before a fix is
> > available for i915 driver.
> >
> > Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> > Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> 
> Wei: I think this is a candidate for inclusion into 4.6
> 

This patch is straight-forward and it's basically risk free. So

Release-acked-by: Wei Liu <wei.liu2@citrix.com>

(subject to an ack from vt-d maintainers)

> ~Andrew
> 
> > ---
> > Changed since v2:
> >   * Make no-igfx available for all Intel devices, not just Calpella/Ironlake
> > Changed since v1:
> >   * Replace igfx_off with igfx
> >
> >  docs/misc/xen-command-line.markdown  | 11 ++++++++++-
> >  xen/drivers/passthrough/iommu.c      |  3 +++
> >  xen/drivers/passthrough/vtd/quirks.c |  3 +++
> >  xen/include/xen/iommu.h              |  2 +-
> >  4 files changed, 17 insertions(+), 2 deletions(-)
> >
> > diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
> > index 13f03ad..486e53b 100644
> > --- a/docs/misc/xen-command-line.markdown
> > +++ b/docs/misc/xen-command-line.markdown
> > @@ -793,7 +793,7 @@ debug hypervisor only).
> >  > Default: `new` unless directed-EOI is supported
> >  
> >  ### iommu
> > -> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | verbose | debug ]`
> > +> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | igfx | verbose | debug ]`
> >  
> >  > Sub-options:
> >  
> > @@ -867,6 +867,15 @@ debug hypervisor only).
> >  >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
> >  >> entry aren't PCI discoverable).
> >  
> > +> `igfx` (VT-d)
> > +
> > +> Default: `true`
> > +
> > +>> Enable IOMMU for Intel graphics devices. The intended usage of this option
> > +>> is `no-igfx`, which is silimar to Linux `intel_iommu=igfx_off` option used
                               ^
                               similar

Wei.

[PATCH v4] VT-d: add iommu=igfx option to workaround graphics issues

[Ting-Wei Lan]

When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
devices, It is possible to encounter graphics issues that make screen
unreadable or crash the system. It was reported in freedesktop bugzilla:

https://bugs.freedesktop.org/show_bug.cgi?id=90037

As we still cannot find a proper fix for this problem, this patch adds
iommu=igfx option to control whether Intel graphics IOMMU is enabled.
Running Xen with iommu=no-igfx is similar to running Linux with
intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
used by users to manually workaround the problem before a fix is
available for i915 driver.

Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
Release-acked-by: Wei Liu <wei.liu2@citrix.com>
---
Changed since v3:
  * Fix typo in xen-command-line.markdown
Changed since v2:
  * Make no-igfx available for all Intel devices, not just Calpella/Ironlake
Changed since v1:
  * Replace igfx_off with igfx

 docs/misc/xen-command-line.markdown  | 11 ++++++++++-
 xen/drivers/passthrough/iommu.c      |  3 +++
 xen/drivers/passthrough/vtd/quirks.c |  3 +++
 xen/include/xen/iommu.h              |  2 +-
 4 files changed, 17 insertions(+), 2 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 13f03ad..a01292a 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -793,7 +793,7 @@ debug hypervisor only).
 > Default: `new` unless directed-EOI is supported
 
 ### iommu
-> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | verbose | debug ]`
+> `= List of [ <boolean> | force | required | intremap | qinval | snoop | sharept | dom0-passthrough | dom0-strict | amd-iommu-perdev-intremap | workaround_bios_bug | igfx | verbose | debug ]`
 
 > Sub-options:
 
@@ -867,6 +867,15 @@ debug hypervisor only).
 >> ignored (normally IOMMU setup fails if any of the devices listed by a DRHD
 >> entry aren't PCI discoverable).
 
+> `igfx` (VT-d)
+
+> Default: `true`
+
+>> Enable IOMMU for Intel graphics devices. The intended usage of this option
+>> is `no-igfx`, which is similar to Linux `intel_iommu=igfx_off` option used
+>> to workaround graphics issues. If adding `no-igfx` fixes anything, you
+>> should file a bug reporting the problem.
+
 > `verbose`
 
 > Default: `false`
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index cc12735..966cc66 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -47,6 +47,7 @@ bool_t __read_mostly force_iommu;
 bool_t __hwdom_initdata iommu_dom0_strict;
 bool_t __read_mostly iommu_verbose;
 bool_t __read_mostly iommu_workaround_bios_bug;
+bool_t __read_mostly iommu_igfx = 1;
 bool_t __read_mostly iommu_passthrough;
 bool_t __read_mostly iommu_snoop = 1;
 bool_t __read_mostly iommu_qinval = 1;
@@ -87,6 +88,8 @@ static void __init parse_iommu_param(char *s)
             force_iommu = val;
         else if ( !strcmp(s, "workaround_bios_bug") )
             iommu_workaround_bios_bug = val;
+        else if ( !strcmp(s, "igfx") )
+            iommu_igfx = val;
         else if ( !strcmp(s, "verbose") )
             iommu_verbose = val;
         else if ( !strcmp(s, "snoop") )
diff --git a/xen/drivers/passthrough/vtd/quirks.c b/xen/drivers/passthrough/vtd/quirks.c
index 69d29ab..92cff1d 100644
--- a/xen/drivers/passthrough/vtd/quirks.c
+++ b/xen/drivers/passthrough/vtd/quirks.c
@@ -72,6 +72,9 @@ int is_igd_vt_enabled_quirk(void)
 {
     u16 ggc;
 
+    if ( !iommu_igfx )
+        return 0;
+
     if ( !IS_ILK(ioh_id) )
         return 1;
 
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 8eb764a..29eed51 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -29,7 +29,7 @@
 
 extern bool_t iommu_enable, iommu_enabled;
 extern bool_t force_iommu, iommu_verbose;
-extern bool_t iommu_workaround_bios_bug, iommu_passthrough;
+extern bool_t iommu_workaround_bios_bug, iommu_igfx, iommu_passthrough;
 extern bool_t iommu_snoop, iommu_qinval, iommu_intremap;
 extern bool_t iommu_hap_pt_share;
 extern bool_t iommu_debug;
-- 
2.4.3

Re: [PATCH v4] VT-d: add iommu=igfx option to workaround graphics issues

[Tian, Kevin]

> From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> Sent: Thursday, August 06, 2015 1:10 AM
> 
> When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> devices, It is possible to encounter graphics issues that make screen
> unreadable or crash the system. It was reported in freedesktop bugzilla:
> 
> https://bugs.freedesktop.org/show_bug.cgi?id=90037
> 
> As we still cannot find a proper fix for this problem, this patch adds
> iommu=igfx option to control whether Intel graphics IOMMU is enabled.
> Running Xen with iommu=no-igfx is similar to running Linux with
> intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
> used by users to manually workaround the problem before a fix is
> available for i915 driver.
> 
> Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> Release-acked-by: Wei Liu <wei.liu2@citrix.com>

Acked-by: Kevin Tian <kevin.tian@intel.com>

Re: [PATCH v4] VT-d: add iommu=igfx option to workaround graphics issues

[Wei Liu]

On Thu, Aug 06, 2015 at 12:49:51AM +0000, Tian, Kevin wrote:
> > From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> > Sent: Thursday, August 06, 2015 1:10 AM
> > 
> > When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel Ironlake
> > devices, It is possible to encounter graphics issues that make screen
> > unreadable or crash the system. It was reported in freedesktop bugzilla:
> > 
> > https://bugs.freedesktop.org/show_bug.cgi?id=90037
> > 
> > As we still cannot find a proper fix for this problem, this patch adds
> > iommu=igfx option to control whether Intel graphics IOMMU is enabled.
> > Running Xen with iommu=no-igfx is similar to running Linux with
> > intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
> > used by users to manually workaround the problem before a fix is
> > available for i915 driver.
> > 
> > Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> > Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > Release-acked-by: Wei Liu <wei.liu2@citrix.com>
> 
> Acked-by: Kevin Tian <kevin.tian@intel.com>

Thanks for the quick turnaround!

Wei.

Re: [PATCH v4] VT-d: add iommu=igfx option to workaround graphics issues

[Ian Campbell]

On Thu, 2015-08-06 at 09:25 +0100, Wei Liu wrote:
> On Thu, Aug 06, 2015 at 12:49:51AM +0000, Tian, Kevin wrote:
> > > From: Ting-Wei Lan [mailto:lantw44@gmail.com]
> > > Sent: Thursday, August 06, 2015 1:10 AM
> > > 
> > > When using Linux >= 3.19 (commit 47591df) as dom0 on some Intel 
> > > Ironlake
> > > devices, It is possible to encounter graphics issues that make screen
> > > unreadable or crash the system. It was reported in freedesktop 
> > > bugzilla:
> > > 
> > > https://bugs.freedesktop.org/show_bug.cgi?id=90037
> > > 
> > > As we still cannot find a proper fix for this problem, this patch 
> > > adds
> > > iommu=igfx option to control whether Intel graphics IOMMU is enabled.
> > > Running Xen with iommu=no-igfx is similar to running Linux with
> > > intel_iommu=igfx_off, which disables IOMMU for Intel GPU. This can be
> > > used by users to manually workaround the problem before a fix is
> > > available for i915 driver.
> > > 
> > > Signed-off-by: Ting-Wei Lan <lantw44@gmail.com>
> > > Reviewed-by: Andrew Cooper <andrew.cooper3@citrix.com>
> > > Release-acked-by: Wei Liu <wei.liu2@citrix.com>
> > 
> > Acked-by: Kevin Tian <kevin.tian@intel.com>
> 
> Thanks for the quick turnaround!

Applied.