[PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Roger Pau Monne]

Hello,

The following series implement a workaround for missing RMRR
entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
option.

The PVH workaround identity maps all regions marked as reserved in the
memory map.

Note that this workaround is enabled by default on Intel hardware. It's
also available to AMD hardware, although it's disabled by default in
that case.

The series can be found at:

git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3

Thanks, Roger.
Roger Pau Monne (4):
  iommu: introduce dom0-iommu option
  iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
  dom0/pvh: change the order of the MMCFG initialization
  x86/iommu: add reserved dom0-iommu option to map reserved memory
    ranges

 docs/misc/xen-command-line.markdown         | 47 +++++++++++
 xen/arch/x86/hvm/dom0_build.c               |  9 ++-
 xen/arch/x86/hvm/io.c                       |  5 ++
 xen/arch/x86/x86_64/mm.c                    |  3 +-
 xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
 xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
 xen/drivers/passthrough/arm/iommu.c         |  4 +
 xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
 xen/drivers/passthrough/vtd/extern.h        |  2 -
 xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
 xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
 xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
 xen/include/asm-x86/hvm/io.h                |  3 +
 xen/include/xen/iommu.h                     |  8 +-
 14 files changed, 240 insertions(+), 86 deletions(-)

-- 
2.18.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 1/4] iommu: introduce dom0-iommu option

[Roger Pau Monne]

To select the iommu configuration used by Dom0. This option supersedes
iommu=dom0-strict|dom0-passthrough.

No functional change.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
Reviewed-by: Paul Durrant <paul.durrant@citrix.com>
---
Changes since v2:
 - Change the style and text used in the xen command line document.
 - Don't allow none, strict or relaxed to use the no- prefix.

Changes since v1:
 - New in this version.
---
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien.grall@arm.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Tim Deegan <tim@xen.org>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: Brian Woods <brian.woods@amd.com>
Cc: Kevin Tian <kevin.tian@intel.com>
---
 docs/misc/xen-command-line.markdown         | 23 +++++++++++
 xen/arch/x86/x86_64/mm.c                    |  3 +-
 xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  4 +-
 xen/drivers/passthrough/iommu.c             | 42 +++++++++++++++++----
 xen/drivers/passthrough/vtd/iommu.c         | 16 ++++----
 xen/include/xen/iommu.h                     |  6 ++-
 7 files changed, 75 insertions(+), 21 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 65b4754418..ea451f088e 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1150,12 +1150,18 @@ detection of systems known to misbehave upon accesses to that port.
 
 > `dom0-passthrough`
 
+> **WARNING: This command line option is deprecated, and superseded by
+> _dom0-iommu=none_ - using both options in combination is undefined.**
+
 > Default: `false`
 
 >> Control whether to disable DMA remapping for Dom0.
 
 > `dom0-strict`
 
+> **WARNING: This command line option is deprecated, and superseded by
+> _dom0-iommu=strict_ - using both options in combination is undefined.**
+
 > Default: `false`
 
 >> Control whether to set up DMA remapping only for the memory Dom0 actually
@@ -1198,6 +1204,23 @@ detection of systems known to misbehave upon accesses to that port.
 
 >> Enable IOMMU debugging code (implies `verbose`).
 
+### dom0-iommu
+> `= List of [ none | strict | relaxed ]`
+
+* `none`: disables DMA remapping for Dom0.
+
+The following two options control how RAM regions are mapped in the iommu for
+PV Dom0:
+
+* `strict`: sets up DMA remapping only for the memory Dom0 actually got
+  assigned.
+
+* `relaxed`: sets DMA remapping for all the host RAM except regions in use by
+  Xen. This is the default iommu behaviour.
+
+Note that all the above options are mutually exclusive. Specifying more than
+one on the `dom0-iommu` command line will result in undefined behavior.
+
 ### iommu\_dev\_iotlb\_timeout
 > `= <integer>`
 
diff --git a/xen/arch/x86/x86_64/mm.c b/xen/arch/x86/x86_64/mm.c
index cca4ae926e..84226b3326 100644
--- a/xen/arch/x86/x86_64/mm.c
+++ b/xen/arch/x86/x86_64/mm.c
@@ -1426,7 +1426,8 @@ int memory_add(unsigned long spfn, unsigned long epfn, unsigned int pxm)
     if ( ret )
         goto destroy_m2p;
 
-    if ( iommu_enabled && !iommu_passthrough && !need_iommu(hardware_domain) )
+    if ( iommu_enabled && !iommu_dom0_passthrough &&
+         !need_iommu(hardware_domain) )
     {
         for ( i = spfn; i < epfn; i++ )
             if ( iommu_map_page(hardware_domain, i, i, IOMMUF_readable|IOMMUF_writable) )
diff --git a/xen/drivers/passthrough/amd/iommu_init.c b/xen/drivers/passthrough/amd/iommu_init.c
index 474992a75a..ad8c48be1c 100644
--- a/xen/drivers/passthrough/amd/iommu_init.c
+++ b/xen/drivers/passthrough/amd/iommu_init.c
@@ -1062,7 +1062,7 @@ static void __init amd_iommu_init_cleanup(void)
     radix_tree_destroy(&ivrs_maps, xfree);
 
     iommu_enabled = 0;
-    iommu_passthrough = 0;
+    iommu_dom0_passthrough = false;
     iommu_intremap = 0;
     iommuv2_enabled = 0;
 }
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 12d2695b89..eeacf713e4 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -121,7 +121,7 @@ static void amd_iommu_setup_domain_device(
     BUG_ON( !hd->arch.root_table || !hd->arch.paging_mode ||
             !iommu->dev_table.buffer );
 
-    if ( iommu_passthrough && is_hardware_domain(domain) )
+    if ( iommu_dom0_passthrough && is_hardware_domain(domain) )
         valid = 0;
 
     if ( ats_enabled )
@@ -256,7 +256,7 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
     if ( allocate_domain_resources(dom_iommu(d)) )
         BUG();
 
-    if ( !iommu_passthrough && !need_iommu(d) )
+    if ( !iommu_dom0_passthrough && !need_iommu(d) )
     {
         int rc = 0;
 
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 70d218f910..830560bdcf 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -22,6 +22,7 @@
 #include <xsm/xsm.h>
 
 static int parse_iommu_param(const char *s);
+static int parse_dom0_iommu_param(const char *s);
 static void iommu_dump_p2m_table(unsigned char key);
 
 unsigned int __read_mostly iommu_dev_iotlb_timeout = 1000;
@@ -52,11 +53,9 @@ custom_param("iommu", parse_iommu_param);
 bool_t __initdata iommu_enable = 1;
 bool_t __read_mostly iommu_enabled;
 bool_t __read_mostly force_iommu;
-bool_t __hwdom_initdata iommu_dom0_strict;
 bool_t __read_mostly iommu_verbose;
 bool_t __read_mostly iommu_workaround_bios_bug;
 bool_t __read_mostly iommu_igfx = 1;
-bool_t __read_mostly iommu_passthrough;
 bool_t __read_mostly iommu_snoop = 1;
 bool_t __read_mostly iommu_qinval = 1;
 bool_t __read_mostly iommu_intremap = 1;
@@ -72,6 +71,10 @@ bool_t __read_mostly iommu_hap_pt_share = 1;
 bool_t __read_mostly iommu_debug;
 bool_t __read_mostly amd_iommu_perdev_intremap = 1;
 
+custom_param("dom0-iommu", parse_dom0_iommu_param);
+bool __hwdom_initdata iommu_dom0_strict;
+bool __read_mostly iommu_dom0_passthrough;
+
 DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
 
 DEFINE_SPINLOCK(iommu_pt_cleanup_lock);
@@ -121,7 +124,7 @@ static int __init parse_iommu_param(const char *s)
         else if ( !strncmp(s, "amd-iommu-perdev-intremap", ss - s) )
             amd_iommu_perdev_intremap = val;
         else if ( !strncmp(s, "dom0-passthrough", ss - s) )
-            iommu_passthrough = val;
+            iommu_dom0_passthrough = val;
         else if ( !strncmp(s, "dom0-strict", ss - s) )
             iommu_dom0_strict = val;
         else if ( !strncmp(s, "sharept", ss - s) )
@@ -135,6 +138,31 @@ static int __init parse_iommu_param(const char *s)
     return rc;
 }
 
+static int __init parse_dom0_iommu_param(const char *s)
+{
+    const char *ss;
+    int rc = 0;
+
+    do {
+        ss = strchr(s, ',');
+        if ( !ss )
+            ss = strchr(s, '\0');
+
+        if ( !strncmp(s, "none", ss - s) )
+            iommu_dom0_passthrough = true;
+        else if ( !strncmp(s, "strict", ss - s) )
+            iommu_dom0_strict = true;
+        else if ( !strncmp(s, "relaxed", ss - s) )
+            iommu_dom0_strict = false;
+        else
+            rc = -EINVAL;
+
+        s = ss + 1;
+    } while ( *ss );
+
+    return rc;
+}
+
 int iommu_domain_init(struct domain *d)
 {
     struct domain_iommu *hd = dom_iommu(d);
@@ -158,7 +186,7 @@ static void __hwdom_init check_hwdom_reqs(struct domain *d)
 
     arch_iommu_check_autotranslated_hwdom(d);
 
-    if ( iommu_passthrough )
+    if ( iommu_dom0_passthrough )
         panic("Dom0 uses paging translated mode, dom0-passthrough must not be "
               "enabled\n");
 
@@ -372,7 +400,7 @@ int __init iommu_setup(void)
     bool_t force_intremap = force_iommu && iommu_intremap;
 
     if ( iommu_dom0_strict )
-        iommu_passthrough = 0;
+        iommu_dom0_passthrough = false;
 
     if ( iommu_enable )
     {
@@ -393,14 +421,14 @@ int __init iommu_setup(void)
     if ( !iommu_enabled )
     {
         iommu_snoop = 0;
-        iommu_passthrough = 0;
+        iommu_dom0_passthrough = false;
         iommu_dom0_strict = 0;
     }
     printk("I/O virtualisation %sabled\n", iommu_enabled ? "en" : "dis");
     if ( iommu_enabled )
     {
         printk(" - Dom0 mode: %s\n",
-               iommu_passthrough ? "Passthrough" :
+               iommu_dom0_passthrough ? "Passthrough" :
                iommu_dom0_strict ? "Strict" : "Relaxed");
         printk("Interrupt remapping %sabled\n", iommu_intremap ? "en" : "dis");
         tasklet_init(&iommu_pt_cleanup_tasklet, iommu_free_pagetables, 0);
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 1710256823..8ac774215b 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1304,7 +1304,7 @@ static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
 {
     struct acpi_drhd_unit *drhd;
 
-    if ( !iommu_passthrough && is_pv_domain(d) )
+    if ( !iommu_dom0_passthrough && is_pv_domain(d) )
     {
         /* Set up 1:1 page table for hardware domain. */
         vtd_set_hwdom_mapping(d);
@@ -1391,7 +1391,7 @@ int domain_context_mapping_one(
         return res;
     }
 
-    if ( iommu_passthrough && is_hardware_domain(domain) )
+    if ( iommu_dom0_passthrough && is_hardware_domain(domain) )
     {
         context_set_translation_type(*context, CONTEXT_TT_PASS_THRU);
         agaw = level_to_agaw(iommu->nr_pt_levels);
@@ -1781,7 +1781,7 @@ static int __must_check intel_iommu_map_page(struct domain *d,
         return 0;
 
     /* Do nothing if hardware domain and iommu supports pass thru. */
-    if ( iommu_passthrough && is_hardware_domain(d) )
+    if ( iommu_dom0_passthrough && is_hardware_domain(d) )
         return 0;
 
     spin_lock(&hd->arch.mapping_lock);
@@ -1826,7 +1826,7 @@ static int __must_check intel_iommu_unmap_page(struct domain *d,
                                                unsigned long gfn)
 {
     /* Do nothing if hardware domain and iommu supports pass thru. */
-    if ( iommu_passthrough && is_hardware_domain(d) )
+    if ( iommu_dom0_passthrough && is_hardware_domain(d) )
         return 0;
 
     return dma_pte_clear_one(d, (paddr_t)gfn << PAGE_SHIFT_4K);
@@ -2269,8 +2269,8 @@ int __init intel_vtd_setup(void)
         if ( iommu_snoop && !ecap_snp_ctl(iommu->ecap) )
             iommu_snoop = 0;
 
-        if ( iommu_passthrough && !ecap_pass_thru(iommu->ecap) )
-            iommu_passthrough = 0;
+        if ( iommu_dom0_passthrough && !ecap_pass_thru(iommu->ecap) )
+            iommu_dom0_passthrough = false;
 
         if ( iommu_qinval && !ecap_queued_inval(iommu->ecap) )
             iommu_qinval = 0;
@@ -2308,7 +2308,7 @@ int __init intel_vtd_setup(void)
 
 #define P(p,s) printk("Intel VT-d %s %senabled.\n", s, (p)? "" : "not ")
     P(iommu_snoop, "Snoop Control");
-    P(iommu_passthrough, "Dom0 DMA Passthrough");
+    P(iommu_dom0_passthrough, "Dom0 DMA Passthrough");
     P(iommu_qinval, "Queued Invalidation");
     P(iommu_intremap, "Interrupt Remapping");
     P(iommu_intpost, "Posted Interrupt");
@@ -2330,7 +2330,7 @@ int __init intel_vtd_setup(void)
  error:
     iommu_enabled = 0;
     iommu_snoop = 0;
-    iommu_passthrough = 0;
+    iommu_dom0_passthrough = false;
     iommu_qinval = 0;
     iommu_intremap = 0;
     iommu_intpost = 0;
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 6b42e3b876..c0c6975ac4 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -29,13 +29,15 @@
 #include <asm/iommu.h>
 
 extern bool_t iommu_enable, iommu_enabled;
-extern bool_t force_iommu, iommu_dom0_strict, iommu_verbose;
-extern bool_t iommu_workaround_bios_bug, iommu_igfx, iommu_passthrough;
+extern bool_t force_iommu, iommu_verbose;
+extern bool_t iommu_workaround_bios_bug, iommu_igfx;
 extern bool_t iommu_snoop, iommu_qinval, iommu_intremap, iommu_intpost;
 extern bool_t iommu_hap_pt_share;
 extern bool_t iommu_debug;
 extern bool_t amd_iommu_perdev_intremap;
 
+extern bool iommu_dom0_strict, iommu_dom0_passthrough;
+
 extern unsigned int iommu_dev_iotlb_timeout;
 
 int iommu_setup(void);
-- 
2.18.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 2/4] iommu: make iommu_inclusive_mapping a suboption of dom0-iommu

[Roger Pau Monne]

Introduce a new dom0-iommu=inclusive generic option that supersedes
iommu_inclusive_mapping. The previous behaviour is preserved and the
option should only be enabled by default on Intel hardware.

No functional change intended.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Fix typo in commit message.
 - Change style and text of the documentation in xen command line.
 - Set the defaults in {intel/amd}_iommu_hwdom_init for inclusive.
 - Re-add the iommu_dom0_passthrough || !is_pv_domain(d) check.

Changes since v1:
 - Use dom0-iommu instead of the iommu option.
 - Only enable by default on Intel hardware.
---
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien.grall@arm.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Tim Deegan <tim@xen.org>
Cc: Wei Liu <wei.liu2@citrix.com>
Cc: Kevin Tian <kevin.tian@intel.com>
---
 docs/misc/xen-command-line.markdown         | 17 +++++-
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  4 ++
 xen/drivers/passthrough/arm/iommu.c         |  4 ++
 xen/drivers/passthrough/iommu.c             | 23 ++++++--
 xen/drivers/passthrough/vtd/extern.h        |  2 -
 xen/drivers/passthrough/vtd/iommu.c         |  8 ++-
 xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------------
 xen/drivers/passthrough/x86/iommu.c         | 59 +++++++++++++++++++++
 xen/include/xen/iommu.h                     |  2 +
 9 files changed, 109 insertions(+), 68 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index ea451f088e..90b32fe3f0 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon accesses to that port.
 >> Enable IOMMU debugging code (implies `verbose`).
 
 ### dom0-iommu
-> `= List of [ none | strict | relaxed ]`
+> `= List of [ none | strict | relaxed | inclusive ]`
 
 * `none`: disables DMA remapping for Dom0.
 
@@ -1221,6 +1221,18 @@ PV Dom0:
 Note that all the above options are mutually exclusive. Specifying more than
 one on the `dom0-iommu` command line will result in undefined behavior.
 
+The following options control whether non-RAM regions are added to the Dom0
+iommu tables. Note that they can be prefixed with `no-` to effect the inverse
+meaning:
+
+* `inclusive`: sets up DMA remapping for all the non-RAM memory below 4GB
+  except for unusable ranges. Use this to work around firmware issues providing
+  incorrect RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU
+  accesses for Dom0, with this option all pages up to 4GB, not marked as
+  unusable in the E820 table, will get a mapping established. Note that this
+  option is only applicable to a PV Dom0 and is enabled by default on Intel
+  hardware.
+
 ### iommu\_dev\_iotlb\_timeout
 > `= <integer>`
 
@@ -1233,6 +1245,9 @@ wait descriptor timed out', try increasing this value.
 ### iommu\_inclusive\_mapping (VT-d)
 > `= <boolean>`
 
+**WARNING: This command line option is deprecated, and superseded by
+_dom0-iommu=inclusive_ - using both options in combination is undefined.**
+
 > Default: `true`
 
 Use this to work around firmware issues providing incorrect RMRR entries.
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index eeacf713e4..0e0c99c942 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -253,6 +253,10 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
     unsigned long i; 
     const struct amd_iommu *iommu;
 
+    /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
+    iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
+                                                      : iommu_dom0_inclusive;
+
     if ( allocate_domain_resources(dom_iommu(d)) )
         BUG();
 
diff --git a/xen/drivers/passthrough/arm/iommu.c b/xen/drivers/passthrough/arm/iommu.c
index 95b1abb972..325997b19f 100644
--- a/xen/drivers/passthrough/arm/iommu.c
+++ b/xen/drivers/passthrough/arm/iommu.c
@@ -73,3 +73,7 @@ int arch_iommu_populate_page_table(struct domain *d)
     /* The IOMMU shares the p2m with the CPU */
     return -ENOSYS;
 }
+
+void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
+{
+}
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index 830560bdcf..f15c94be42 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -74,6 +74,7 @@ bool_t __read_mostly amd_iommu_perdev_intremap = 1;
 custom_param("dom0-iommu", parse_dom0_iommu_param);
 bool __hwdom_initdata iommu_dom0_strict;
 bool __read_mostly iommu_dom0_passthrough;
+int8_t __hwdom_initdata iommu_dom0_inclusive = -1;
 
 DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
 
@@ -144,16 +145,23 @@ static int __init parse_dom0_iommu_param(const char *s)
     int rc = 0;
 
     do {
+        bool val = !!strncmp(s, "no-", 3);
+
+        if ( !val )
+            s += 3;
+
         ss = strchr(s, ',');
         if ( !ss )
             ss = strchr(s, '\0');
 
-        if ( !strncmp(s, "none", ss - s) )
+        if ( !strncmp(s, "none", ss - s) && val )
             iommu_dom0_passthrough = true;
-        else if ( !strncmp(s, "strict", ss - s) )
+        else if ( !strncmp(s, "strict", ss - s) && val )
             iommu_dom0_strict = true;
-        else if ( !strncmp(s, "relaxed", ss - s) )
+        else if ( !strncmp(s, "relaxed", ss - s) && val )
             iommu_dom0_strict = false;
+        else if ( !strncmp(s, "inclusive", ss - s) )
+            iommu_dom0_inclusive = val;
         else
             rc = -EINVAL;
 
@@ -202,6 +210,13 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
     if ( !iommu_enabled )
         return;
 
+    if ( iommu_dom0_inclusive == true && !is_pv_domain(d) )
+    {
+        printk(XENLOG_WARNING
+               "IOMMU inclusive mappings are only supported on PV Dom0\n");
+        iommu_dom0_inclusive = false;
+    }
+
     register_keyhandler('o', &iommu_dump_p2m_table, "dump iommu p2m table", 0);
     d->need_iommu = !!iommu_dom0_strict;
     if ( need_iommu(d) && !iommu_use_hap_pt(d) )
@@ -236,6 +251,8 @@ void __hwdom_init iommu_hwdom_init(struct domain *d)
     }
 
     hd->platform_ops->hwdom_init(d);
+
+    arch_iommu_hwdom_init(d);
 }
 
 void iommu_teardown(struct domain *d)
diff --git a/xen/drivers/passthrough/vtd/extern.h b/xen/drivers/passthrough/vtd/extern.h
index fb7edfaef9..91cadc602e 100644
--- a/xen/drivers/passthrough/vtd/extern.h
+++ b/xen/drivers/passthrough/vtd/extern.h
@@ -99,6 +99,4 @@ void pci_vtd_quirk(const struct pci_dev *);
 bool_t platform_supports_intremap(void);
 bool_t platform_supports_x2apic(void);
 
-void vtd_set_hwdom_mapping(struct domain *d);
-
 #endif // _VTD_EXTERN_H_
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 8ac774215b..7c7e15755d 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1304,11 +1304,9 @@ static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
 {
     struct acpi_drhd_unit *drhd;
 
-    if ( !iommu_dom0_passthrough && is_pv_domain(d) )
-    {
-        /* Set up 1:1 page table for hardware domain. */
-        vtd_set_hwdom_mapping(d);
-    }
+    /* Inclusive mappings are enabled by default on Intel hardware for PV. */
+    iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? is_pv_domain(d)
+                                                      : iommu_dom0_inclusive;
 
     setup_hwdom_pci_devices(d, setup_hwdom_device);
     setup_hwdom_rmrr(d);
diff --git a/xen/drivers/passthrough/vtd/x86/vtd.c b/xen/drivers/passthrough/vtd/x86/vtd.c
index 00a9891005..20323051d0 100644
--- a/xen/drivers/passthrough/vtd/x86/vtd.c
+++ b/xen/drivers/passthrough/vtd/x86/vtd.c
@@ -25,7 +25,6 @@
 #include <xen/irq.h>
 #include <xen/numa.h>
 #include <asm/fixmap.h>
-#include <asm/setup.h>
 #include "../iommu.h"
 #include "../dmar.h"
 #include "../vtd.h"
@@ -35,8 +34,7 @@
  * iommu_inclusive_mapping: when set, all memory below 4GB is included in dom0
  * 1:1 iommu mappings except xen and unusable regions.
  */
-static bool_t __hwdom_initdata iommu_inclusive_mapping = 1;
-boolean_param("iommu_inclusive_mapping", iommu_inclusive_mapping);
+boolean_param("iommu_inclusive_mapping", iommu_dom0_inclusive);
 
 void *map_vtd_domain_page(u64 maddr)
 {
@@ -108,57 +106,3 @@ void hvm_dpci_isairq_eoi(struct domain *d, unsigned int isairq)
     spin_unlock(&d->event_lock);
 }
 
-void __hwdom_init vtd_set_hwdom_mapping(struct domain *d)
-{
-    unsigned long i, top, max_pfn;
-
-    BUG_ON(!is_hardware_domain(d));
-
-    max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
-    top = max(max_pdx, pfn_to_pdx(max_pfn) + 1);
-
-    for ( i = 0; i < top; i++ )
-    {
-        unsigned long pfn = pdx_to_pfn(i);
-        bool map;
-        int rc;
-
-        /*
-         * Set up 1:1 mapping for dom0. Default to include only
-         * conventional RAM areas and let RMRRs include needed reserved
-         * regions. When set, the inclusive mapping additionally maps in
-         * every pfn up to 4GB except those that fall in unusable ranges.
-         */
-        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
-            continue;
-
-        if ( iommu_inclusive_mapping && pfn <= max_pfn )
-            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
-        else
-            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
-
-        if ( !map )
-            continue;
-
-        /* Exclude Xen bits */
-        if ( xen_in_range(pfn) )
-            continue;
-
-        /*
-         * If dom0-strict mode is enabled then exclude conventional RAM
-         * and let the common code map dom0's pages.
-         */
-        if ( iommu_dom0_strict &&
-             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
-            continue;
-
-        rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
-        if ( rc )
-            printk(XENLOG_WARNING VTDPREFIX " d%d: IOMMU mapping failed: %d\n",
-                   d->domain_id, rc);
-
-        if (!(i & 0xfffff))
-            process_pending_softirqs();
-    }
-}
-
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 68182afd91..5a7a765e9d 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -20,6 +20,8 @@
 #include <xen/softirq.h>
 #include <xsm/xsm.h>
 
+#include <asm/setup.h>
+
 void iommu_update_ire_from_apic(
     unsigned int apic, unsigned int reg, unsigned int value)
 {
@@ -132,6 +134,63 @@ void arch_iommu_domain_destroy(struct domain *d)
 {
 }
 
+void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
+{
+    unsigned long i, top, max_pfn;
+
+    BUG_ON(!is_hardware_domain(d));
+
+    if ( iommu_dom0_passthrough || !is_pv_domain(d) )
+        return;
+
+    max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
+    top = max(max_pdx, pfn_to_pdx(max_pfn) + 1);
+
+    for ( i = 0; i < top; i++ )
+    {
+        unsigned long pfn = pdx_to_pfn(i);
+        bool map;
+        int rc;
+
+        /*
+         * Set up 1:1 mapping for dom0. Default to include only
+         * conventional RAM areas and let RMRRs include needed reserved
+         * regions. When set, the inclusive mapping additionally maps in
+         * every pfn up to 4GB except those that fall in unusable ranges.
+         */
+        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
+            continue;
+
+        if ( iommu_dom0_inclusive && pfn <= max_pfn )
+            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
+        else
+            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
+
+        if ( !map )
+            continue;
+
+        /* Exclude Xen bits */
+        if ( xen_in_range(pfn) )
+            continue;
+
+        /*
+         * If dom0-strict mode is enabled then exclude conventional RAM
+         * and let the common code map dom0's pages.
+         */
+        if ( iommu_dom0_strict &&
+             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
+            continue;
+
+        rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
+        if ( rc )
+            printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
+                   d->domain_id, rc);
+
+        if (!(i & 0xfffff))
+            process_pending_softirqs();
+    }
+}
+
 /*
  * Local variables:
  * mode: C
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index c0c6975ac4..99e5b89c0f 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -37,6 +37,7 @@ extern bool_t iommu_debug;
 extern bool_t amd_iommu_perdev_intremap;
 
 extern bool iommu_dom0_strict, iommu_dom0_passthrough;
+extern int8_t iommu_dom0_inclusive;
 
 extern unsigned int iommu_dev_iotlb_timeout;
 
@@ -51,6 +52,7 @@ void arch_iommu_domain_destroy(struct domain *d);
 int arch_iommu_domain_init(struct domain *d);
 int arch_iommu_populate_page_table(struct domain *d);
 void arch_iommu_check_autotranslated_hwdom(struct domain *d);
+void arch_iommu_hwdom_init(struct domain *d);
 
 int iommu_construct(struct domain *d);
 
-- 
2.18.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 3/4] dom0/pvh: change the order of the MMCFG initialization

[Roger Pau Monne]

So it's done before the iommu is initialized. This is required in
order to be able to fetch the MMCFG regions from the domain struct.

No functional change.

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v1:
 - New in this version.
---
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
---
 xen/arch/x86/hvm/dom0_build.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/xen/arch/x86/hvm/dom0_build.c b/xen/arch/x86/hvm/dom0_build.c
index f0cd63b1ec..5065729106 100644
--- a/xen/arch/x86/hvm/dom0_build.c
+++ b/xen/arch/x86/hvm/dom0_build.c
@@ -1100,6 +1100,13 @@ int __init dom0_construct_pvh(struct domain *d, const module_t *image,
         return rc;
     }
 
+    /*
+     * NB: MMCFG initialization needs to be performed before iommu
+     * initialization so the iommu code can fetch the MMCFG regions used by the
+     * domain.
+     */
+    pvh_setup_mmcfg(d);
+
     iommu_hwdom_init(d);
 
     rc = pvh_load_kernel(d, image, image_headroom, initrd, bootstrap_map(image),
@@ -1124,8 +1131,6 @@ int __init dom0_construct_pvh(struct domain *d, const module_t *image,
         return rc;
     }
 
-    pvh_setup_mmcfg(d);
-
     printk("WARNING: PVH is an experimental mode with limited functionality\n");
     return 0;
 }
-- 
2.18.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[PATCH v3 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

[Roger Pau Monne]

Several people have reported hardware issues (malfunctioning USB
controllers) due to iommu page faults on Intel hardware. Those faults
are caused by missing RMRR (VTd) entries in the ACPI tables. Those can
be worked around on VTd hardware by manually adding RMRR entries on
the command line, this is however limited to Intel hardware and quite
cumbersome to do.

In order to solve those issues add a new dom0-iommu=reserved option
that identity maps all regions marked as reserved in the memory map.
Note that regions used by devices emulated by Xen (LAPIC, IO-APIC or
PCIe MCFG regions) are specifically avoided. Note that this option is
available to a PVH Dom0 (as opposed to the inclusive option which only
works for PV Dom0).

Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>
---
Changes since v2:
 - Fix comment regarding dom0-strict.
 - Change documentation style of xen command line.
 - Rename iommu_map to hwdom_iommu_map.
 - Move all the checks to hwdom_iommu_map.

Changes since v1:
 - Introduce a new reserved option instead of abusing the inclusive
   option.
 - Use the same helper function for PV and PVH in order to decide if a
   page should be added to the domain page tables.
 - Use the data inside of the domain struct to detect overlaps with
   emulated MMIO regions.
---
Cc: Andrew Cooper <andrew.cooper3@citrix.com>
Cc: George Dunlap <George.Dunlap@eu.citrix.com>
Cc: Ian Jackson <ian.jackson@eu.citrix.com>
Cc: Jan Beulich <jbeulich@suse.com>
Cc: Julien Grall <julien.grall@arm.com>
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: Stefano Stabellini <sstabellini@kernel.org>
Cc: Tim Deegan <tim@xen.org>
Cc: Wei Liu <wei.liu2@citrix.com>
---
 docs/misc/xen-command-line.markdown         | 11 ++-
 xen/arch/x86/hvm/io.c                       |  5 ++
 xen/drivers/passthrough/amd/pci_amd_iommu.c |  3 +
 xen/drivers/passthrough/iommu.c             |  3 +
 xen/drivers/passthrough/vtd/iommu.c         |  3 +
 xen/drivers/passthrough/x86/iommu.c         | 86 ++++++++++++++-------
 xen/include/asm-x86/hvm/io.h                |  3 +
 xen/include/xen/iommu.h                     |  2 +-
 8 files changed, 85 insertions(+), 31 deletions(-)

diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-command-line.markdown
index 90b32fe3f0..59ec2afc5d 100644
--- a/docs/misc/xen-command-line.markdown
+++ b/docs/misc/xen-command-line.markdown
@@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon accesses to that port.
 >> Enable IOMMU debugging code (implies `verbose`).
 
 ### dom0-iommu
-> `= List of [ none | strict | relaxed | inclusive ]`
+> `= List of [ none | strict | relaxed | inclusive | reserved ]`
 
 * `none`: disables DMA remapping for Dom0.
 
@@ -1233,6 +1233,15 @@ meaning:
   option is only applicable to a PV Dom0 and is enabled by default on Intel
   hardware.
 
+* `reserved`: sets up DMA remapping for all the reserved regions in the memory
+  map for Dom0. Use this to work around firmware issues providing incorrect
+  RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU accesses
+  for Dom0, all memory regions marked as reserved in the memory map that don't
+  overlap with any MMIO region from emulated devices will be identity mapped.
+  This option maps a subset of the memory that would be mapped when using the
+  `inclusive` option. This option is available to a PVH Dom0 and is enabled by
+  default on Intel hardware.
+
 ### iommu\_dev\_iotlb\_timeout
 > `= <integer>`
 
diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
index bf4d8748d3..5e01c33890 100644
--- a/xen/arch/x86/hvm/io.c
+++ b/xen/arch/x86/hvm/io.c
@@ -404,6 +404,11 @@ static const struct hvm_mmcfg *vpci_mmcfg_find(const struct domain *d,
     return NULL;
 }
 
+bool vpci_mmcfg_address(const struct domain *d, paddr_t addr)
+{
+    return vpci_mmcfg_find(d, addr);
+}
+
 static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg *mmcfg,
                                            paddr_t addr, pci_sbdf_t *sbdf)
 {
diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c b/xen/drivers/passthrough/amd/pci_amd_iommu.c
index 0e0c99c942..2c2867d088 100644
--- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
+++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
@@ -256,6 +256,9 @@ static void __hwdom_init amd_iommu_hwdom_init(struct domain *d)
     /* Inclusive IOMMU mappings are disabled by default on AMD hardware. */
     iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
                                                       : iommu_dom0_inclusive;
+    /* Reserved IOMMU mappings are disabled by default on AMD hardware. */
+    iommu_dom0_reserved = iommu_dom0_reserved == -1 ? false
+                                                    : iommu_dom0_reserved;
 
     if ( allocate_domain_resources(dom_iommu(d)) )
         BUG();
diff --git a/xen/drivers/passthrough/iommu.c b/xen/drivers/passthrough/iommu.c
index f15c94be42..9c991bd2cf 100644
--- a/xen/drivers/passthrough/iommu.c
+++ b/xen/drivers/passthrough/iommu.c
@@ -75,6 +75,7 @@ custom_param("dom0-iommu", parse_dom0_iommu_param);
 bool __hwdom_initdata iommu_dom0_strict;
 bool __read_mostly iommu_dom0_passthrough;
 int8_t __hwdom_initdata iommu_dom0_inclusive = -1;
+int8_t __hwdom_initdata iommu_dom0_reserved = -1;
 
 DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
 
@@ -162,6 +163,8 @@ static int __init parse_dom0_iommu_param(const char *s)
             iommu_dom0_strict = false;
         else if ( !strncmp(s, "inclusive", ss - s) )
             iommu_dom0_inclusive = val;
+        else if ( !strncmp(s, "reserved", ss - s) )
+            iommu_dom0_reserved = val;
         else
             rc = -EINVAL;
 
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index 7c7e15755d..77a076215b 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -1307,6 +1307,9 @@ static void __hwdom_init intel_iommu_hwdom_init(struct domain *d)
     /* Inclusive mappings are enabled by default on Intel hardware for PV. */
     iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? is_pv_domain(d)
                                                       : iommu_dom0_inclusive;
+    /* Reserved IOMMU mappings are enabled by default on Intel hardware. */
+    iommu_dom0_reserved = iommu_dom0_reserved == -1 ? true
+                                                    : iommu_dom0_reserved;
 
     setup_hwdom_pci_devices(d, setup_hwdom_device);
     setup_hwdom_rmrr(d);
diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 5a7a765e9d..6aec43ed1a 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -20,6 +20,7 @@
 #include <xen/softirq.h>
 #include <xsm/xsm.h>
 
+#include <asm/hvm/io.h>
 #include <asm/setup.h>
 
 void iommu_update_ire_from_apic(
@@ -134,13 +135,67 @@ void arch_iommu_domain_destroy(struct domain *d)
 {
 }
 
+static bool __hwdom_init hwdom_iommu_map(const struct domain *d, unsigned long pfn,
+                                         unsigned long max_pfn)
+{
+    unsigned int i;
+
+    /*
+     * Ignore any address below 1MB, that's already identity mapped by the
+     * domain builder for HVM.
+     */
+    if ( (is_hvm_domain(d) && pfn < PFN_DOWN(MB(1))) ||
+         /* Exclude Xen bits. */
+         xen_in_range(pfn) || (pfn > max_pfn && !mfn_valid(_mfn(pfn))) )
+        return false;
+
+    /*
+     * If dom0-strict mode is enabled or the guest type is PVH/HVM then exclude
+     * conventional RAM and let the common code map dom0's pages.
+     */
+    if ( page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
+         (iommu_dom0_strict || is_hvm_domain(d)) )
+        return false;
+    if ( page_is_ram_type(pfn, RAM_TYPE_RESERVED) &&
+         !iommu_dom0_reserved && !iommu_dom0_inclusive )
+        return false;
+    if ( !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE) &&
+         !page_is_ram_type(pfn, RAM_TYPE_RESERVED) &&
+         !page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
+         (!iommu_dom0_inclusive || pfn > max_pfn) )
+        return false;
+
+    /* Check that it doesn't overlap with the LAPIC */
+    if ( has_vlapic(d) )
+    {
+        const struct vcpu *v;
+
+        for_each_vcpu(d, v)
+            if ( pfn == PFN_DOWN(vlapic_base_address(vcpu_vlapic(v))) )
+                return false;
+    }
+    /* ... or the IO-APIC */
+    for ( i = 0; has_vioapic(d) && i < d->arch.hvm_domain.nr_vioapics; i++ )
+        if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) )
+            return false;
+    /*
+     * ... or the PCIe MCFG regions.
+     * TODO: runtime added MMCFG regions are not checked to make sure they
+     * don't overlap with already mapped regions, thus preventing trapping.
+     */
+    if ( has_vpci(d) && vpci_mmcfg_address(d, pfn << PAGE_SHIFT) )
+        return false;
+
+    return true;
+}
+
 void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
 {
     unsigned long i, top, max_pfn;
 
     BUG_ON(!is_hardware_domain(d));
 
-    if ( iommu_dom0_passthrough || !is_pv_domain(d) )
+    if ( iommu_dom0_passthrough )
         return;
 
     max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
@@ -149,36 +204,9 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
     for ( i = 0; i < top; i++ )
     {
         unsigned long pfn = pdx_to_pfn(i);
-        bool map;
         int rc;
 
-        /*
-         * Set up 1:1 mapping for dom0. Default to include only
-         * conventional RAM areas and let RMRRs include needed reserved
-         * regions. When set, the inclusive mapping additionally maps in
-         * every pfn up to 4GB except those that fall in unusable ranges.
-         */
-        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
-            continue;
-
-        if ( iommu_dom0_inclusive && pfn <= max_pfn )
-            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
-        else
-            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
-
-        if ( !map )
-            continue;
-
-        /* Exclude Xen bits */
-        if ( xen_in_range(pfn) )
-            continue;
-
-        /*
-         * If dom0-strict mode is enabled then exclude conventional RAM
-         * and let the common code map dom0's pages.
-         */
-        if ( iommu_dom0_strict &&
-             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
+        if ( !hwdom_iommu_map(d, pfn, max_pfn) )
             continue;
 
         rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h
index e6b6ed0b92..8cca456b55 100644
--- a/xen/include/asm-x86/hvm/io.h
+++ b/xen/include/asm-x86/hvm/io.h
@@ -180,6 +180,9 @@ int register_vpci_mmcfg_handler(struct domain *d, paddr_t addr,
 /* Destroy tracked MMCFG areas. */
 void destroy_vpci_mmcfg(struct domain *d);
 
+/* Check if an address is between a MMCFG region for a domain. */
+bool vpci_mmcfg_address(const struct domain *d, paddr_t addr);
+
 #endif /* __ASM_X86_HVM_IO_H__ */
 
 
diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
index 99e5b89c0f..fed1b1ea7a 100644
--- a/xen/include/xen/iommu.h
+++ b/xen/include/xen/iommu.h
@@ -37,7 +37,7 @@ extern bool_t iommu_debug;
 extern bool_t amd_iommu_perdev_intremap;
 
 extern bool iommu_dom0_strict, iommu_dom0_passthrough;
-extern int8_t iommu_dom0_inclusive;
+extern int8_t iommu_dom0_inclusive, iommu_dom0_reserved;
 
 extern unsigned int iommu_dev_iotlb_timeout;
 
-- 
2.18.0


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 2/4] iommu: make iommu_inclusive_mapping a suboption of dom0-iommu

[Paul Durrant]

> -----Original Message-----
> From: Xen-devel [mailto:xen-devel-bounces@lists.xenproject.org] On Behalf
> Of Roger Pau Monne
> Sent: 07 August 2018 15:03
> To: xen-devel@lists.xenproject.org
> Cc: Kevin Tian <kevin.tian@intel.com>; Stefano Stabellini
> <sstabellini@kernel.org>; Wei Liu <wei.liu2@citrix.com>; George Dunlap
> <George.Dunlap@citrix.com>; Andrew Cooper
> <Andrew.Cooper3@citrix.com>; Ian Jackson <Ian.Jackson@citrix.com>; Tim
> (Xen.org) <tim@xen.org>; Julien Grall <julien.grall@arm.com>; Jan Beulich
> <jbeulich@suse.com>; Roger Pau Monne <roger.pau@citrix.com>
> Subject: [Xen-devel] [PATCH v3 2/4] iommu: make
> iommu_inclusive_mapping a suboption of dom0-iommu
> 
> Introduce a new dom0-iommu=inclusive generic option that supersedes
> iommu_inclusive_mapping. The previous behaviour is preserved and the
> option should only be enabled by default on Intel hardware.
> 
> No functional change intended.
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Paul Durrant <paul.durrant@citrix.com>

> ---
> Changes since v2:
>  - Fix typo in commit message.
>  - Change style and text of the documentation in xen command line.
>  - Set the defaults in {intel/amd}_iommu_hwdom_init for inclusive.
>  - Re-add the iommu_dom0_passthrough || !is_pv_domain(d) check.
> 
> Changes since v1:
>  - Use dom0-iommu instead of the iommu option.
>  - Only enable by default on Intel hardware.
> ---
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: George Dunlap <George.Dunlap@eu.citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Julien Grall <julien.grall@arm.com>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Tim Deegan <tim@xen.org>
> Cc: Wei Liu <wei.liu2@citrix.com>
> Cc: Kevin Tian <kevin.tian@intel.com>
> ---
>  docs/misc/xen-command-line.markdown         | 17 +++++-
>  xen/drivers/passthrough/amd/pci_amd_iommu.c |  4 ++
>  xen/drivers/passthrough/arm/iommu.c         |  4 ++
>  xen/drivers/passthrough/iommu.c             | 23 ++++++--
>  xen/drivers/passthrough/vtd/extern.h        |  2 -
>  xen/drivers/passthrough/vtd/iommu.c         |  8 ++-
>  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------------
>  xen/drivers/passthrough/x86/iommu.c         | 59 +++++++++++++++++++++
>  xen/include/xen/iommu.h                     |  2 +
>  9 files changed, 109 insertions(+), 68 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-
> command-line.markdown
> index ea451f088e..90b32fe3f0 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon
> accesses to that port.
>  >> Enable IOMMU debugging code (implies `verbose`).
> 
>  ### dom0-iommu
> -> `= List of [ none | strict | relaxed ]`
> +> `= List of [ none | strict | relaxed | inclusive ]`
> 
>  * `none`: disables DMA remapping for Dom0.
> 
> @@ -1221,6 +1221,18 @@ PV Dom0:
>  Note that all the above options are mutually exclusive. Specifying more than
>  one on the `dom0-iommu` command line will result in undefined behavior.
> 
> +The following options control whether non-RAM regions are added to the
> Dom0
> +iommu tables. Note that they can be prefixed with `no-` to effect the
> inverse
> +meaning:
> +
> +* `inclusive`: sets up DMA remapping for all the non-RAM memory below
> 4GB
> +  except for unusable ranges. Use this to work around firmware issues
> providing
> +  incorrect RMRR/IVMD entries. Rather than only mapping RAM pages for
> IOMMU
> +  accesses for Dom0, with this option all pages up to 4GB, not marked as
> +  unusable in the E820 table, will get a mapping established. Note that this
> +  option is only applicable to a PV Dom0 and is enabled by default on Intel
> +  hardware.
> +
>  ### iommu\_dev\_iotlb\_timeout
>  > `= <integer>`
> 
> @@ -1233,6 +1245,9 @@ wait descriptor timed out', try increasing this value.
>  ### iommu\_inclusive\_mapping (VT-d)
>  > `= <boolean>`
> 
> +**WARNING: This command line option is deprecated, and superseded by
> +_dom0-iommu=inclusive_ - using both options in combination is
> undefined.**
> +
>  > Default: `true`
> 
>  Use this to work around firmware issues providing incorrect RMRR entries.
> diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> index eeacf713e4..0e0c99c942 100644
> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -253,6 +253,10 @@ static void __hwdom_init
> amd_iommu_hwdom_init(struct domain *d)
>      unsigned long i;
>      const struct amd_iommu *iommu;
> 
> +    /* Inclusive IOMMU mappings are disabled by default on AMD hardware.
> */
> +    iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
> +                                                      : iommu_dom0_inclusive;
> +
>      if ( allocate_domain_resources(dom_iommu(d)) )
>          BUG();
> 
> diff --git a/xen/drivers/passthrough/arm/iommu.c
> b/xen/drivers/passthrough/arm/iommu.c
> index 95b1abb972..325997b19f 100644
> --- a/xen/drivers/passthrough/arm/iommu.c
> +++ b/xen/drivers/passthrough/arm/iommu.c
> @@ -73,3 +73,7 @@ int arch_iommu_populate_page_table(struct domain
> *d)
>      /* The IOMMU shares the p2m with the CPU */
>      return -ENOSYS;
>  }
> +
> +void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
> +{
> +}
> diff --git a/xen/drivers/passthrough/iommu.c
> b/xen/drivers/passthrough/iommu.c
> index 830560bdcf..f15c94be42 100644
> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -74,6 +74,7 @@ bool_t __read_mostly amd_iommu_perdev_intremap
> = 1;
>  custom_param("dom0-iommu", parse_dom0_iommu_param);
>  bool __hwdom_initdata iommu_dom0_strict;
>  bool __read_mostly iommu_dom0_passthrough;
> +int8_t __hwdom_initdata iommu_dom0_inclusive = -1;
> 
>  DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
> 
> @@ -144,16 +145,23 @@ static int __init parse_dom0_iommu_param(const
> char *s)
>      int rc = 0;
> 
>      do {
> +        bool val = !!strncmp(s, "no-", 3);
> +
> +        if ( !val )
> +            s += 3;
> +
>          ss = strchr(s, ',');
>          if ( !ss )
>              ss = strchr(s, '\0');
> 
> -        if ( !strncmp(s, "none", ss - s) )
> +        if ( !strncmp(s, "none", ss - s) && val )
>              iommu_dom0_passthrough = true;
> -        else if ( !strncmp(s, "strict", ss - s) )
> +        else if ( !strncmp(s, "strict", ss - s) && val )
>              iommu_dom0_strict = true;
> -        else if ( !strncmp(s, "relaxed", ss - s) )
> +        else if ( !strncmp(s, "relaxed", ss - s) && val )
>              iommu_dom0_strict = false;
> +        else if ( !strncmp(s, "inclusive", ss - s) )
> +            iommu_dom0_inclusive = val;
>          else
>              rc = -EINVAL;
> 
> @@ -202,6 +210,13 @@ void __hwdom_init iommu_hwdom_init(struct
> domain *d)
>      if ( !iommu_enabled )
>          return;
> 
> +    if ( iommu_dom0_inclusive == true && !is_pv_domain(d) )
> +    {
> +        printk(XENLOG_WARNING
> +               "IOMMU inclusive mappings are only supported on PV Dom0\n");
> +        iommu_dom0_inclusive = false;
> +    }
> +
>      register_keyhandler('o', &iommu_dump_p2m_table, "dump iommu p2m
> table", 0);
>      d->need_iommu = !!iommu_dom0_strict;
>      if ( need_iommu(d) && !iommu_use_hap_pt(d) )
> @@ -236,6 +251,8 @@ void __hwdom_init iommu_hwdom_init(struct
> domain *d)
>      }
> 
>      hd->platform_ops->hwdom_init(d);
> +
> +    arch_iommu_hwdom_init(d);
>  }
> 
>  void iommu_teardown(struct domain *d)
> diff --git a/xen/drivers/passthrough/vtd/extern.h
> b/xen/drivers/passthrough/vtd/extern.h
> index fb7edfaef9..91cadc602e 100644
> --- a/xen/drivers/passthrough/vtd/extern.h
> +++ b/xen/drivers/passthrough/vtd/extern.h
> @@ -99,6 +99,4 @@ void pci_vtd_quirk(const struct pci_dev *);
>  bool_t platform_supports_intremap(void);
>  bool_t platform_supports_x2apic(void);
> 
> -void vtd_set_hwdom_mapping(struct domain *d);
> -
>  #endif // _VTD_EXTERN_H_
> diff --git a/xen/drivers/passthrough/vtd/iommu.c
> b/xen/drivers/passthrough/vtd/iommu.c
> index 8ac774215b..7c7e15755d 100644
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -1304,11 +1304,9 @@ static void __hwdom_init
> intel_iommu_hwdom_init(struct domain *d)
>  {
>      struct acpi_drhd_unit *drhd;
> 
> -    if ( !iommu_dom0_passthrough && is_pv_domain(d) )
> -    {
> -        /* Set up 1:1 page table for hardware domain. */
> -        vtd_set_hwdom_mapping(d);
> -    }
> +    /* Inclusive mappings are enabled by default on Intel hardware for PV. */
> +    iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ?
> is_pv_domain(d)
> +                                                      : iommu_dom0_inclusive;
> 
>      setup_hwdom_pci_devices(d, setup_hwdom_device);
>      setup_hwdom_rmrr(d);
> diff --git a/xen/drivers/passthrough/vtd/x86/vtd.c
> b/xen/drivers/passthrough/vtd/x86/vtd.c
> index 00a9891005..20323051d0 100644
> --- a/xen/drivers/passthrough/vtd/x86/vtd.c
> +++ b/xen/drivers/passthrough/vtd/x86/vtd.c
> @@ -25,7 +25,6 @@
>  #include <xen/irq.h>
>  #include <xen/numa.h>
>  #include <asm/fixmap.h>
> -#include <asm/setup.h>
>  #include "../iommu.h"
>  #include "../dmar.h"
>  #include "../vtd.h"
> @@ -35,8 +34,7 @@
>   * iommu_inclusive_mapping: when set, all memory below 4GB is included in
> dom0
>   * 1:1 iommu mappings except xen and unusable regions.
>   */
> -static bool_t __hwdom_initdata iommu_inclusive_mapping = 1;
> -boolean_param("iommu_inclusive_mapping", iommu_inclusive_mapping);
> +boolean_param("iommu_inclusive_mapping", iommu_dom0_inclusive);
> 
>  void *map_vtd_domain_page(u64 maddr)
>  {
> @@ -108,57 +106,3 @@ void hvm_dpci_isairq_eoi(struct domain *d,
> unsigned int isairq)
>      spin_unlock(&d->event_lock);
>  }
> 
> -void __hwdom_init vtd_set_hwdom_mapping(struct domain *d)
> -{
> -    unsigned long i, top, max_pfn;
> -
> -    BUG_ON(!is_hardware_domain(d));
> -
> -    max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
> -    top = max(max_pdx, pfn_to_pdx(max_pfn) + 1);
> -
> -    for ( i = 0; i < top; i++ )
> -    {
> -        unsigned long pfn = pdx_to_pfn(i);
> -        bool map;
> -        int rc;
> -
> -        /*
> -         * Set up 1:1 mapping for dom0. Default to include only
> -         * conventional RAM areas and let RMRRs include needed reserved
> -         * regions. When set, the inclusive mapping additionally maps in
> -         * every pfn up to 4GB except those that fall in unusable ranges.
> -         */
> -        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
> -            continue;
> -
> -        if ( iommu_inclusive_mapping && pfn <= max_pfn )
> -            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
> -        else
> -            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
> -
> -        if ( !map )
> -            continue;
> -
> -        /* Exclude Xen bits */
> -        if ( xen_in_range(pfn) )
> -            continue;
> -
> -        /*
> -         * If dom0-strict mode is enabled then exclude conventional RAM
> -         * and let the common code map dom0's pages.
> -         */
> -        if ( iommu_dom0_strict &&
> -             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
> -            continue;
> -
> -        rc = iommu_map_page(d, pfn, pfn,
> IOMMUF_readable|IOMMUF_writable);
> -        if ( rc )
> -            printk(XENLOG_WARNING VTDPREFIX " d%d: IOMMU mapping failed:
> %d\n",
> -                   d->domain_id, rc);
> -
> -        if (!(i & 0xfffff))
> -            process_pending_softirqs();
> -    }
> -}
> -
> diff --git a/xen/drivers/passthrough/x86/iommu.c
> b/xen/drivers/passthrough/x86/iommu.c
> index 68182afd91..5a7a765e9d 100644
> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -20,6 +20,8 @@
>  #include <xen/softirq.h>
>  #include <xsm/xsm.h>
> 
> +#include <asm/setup.h>
> +
>  void iommu_update_ire_from_apic(
>      unsigned int apic, unsigned int reg, unsigned int value)
>  {
> @@ -132,6 +134,63 @@ void arch_iommu_domain_destroy(struct domain
> *d)
>  {
>  }
> 
> +void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
> +{
> +    unsigned long i, top, max_pfn;
> +
> +    BUG_ON(!is_hardware_domain(d));
> +
> +    if ( iommu_dom0_passthrough || !is_pv_domain(d) )
> +        return;
> +
> +    max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
> +    top = max(max_pdx, pfn_to_pdx(max_pfn) + 1);
> +
> +    for ( i = 0; i < top; i++ )
> +    {
> +        unsigned long pfn = pdx_to_pfn(i);
> +        bool map;
> +        int rc;
> +
> +        /*
> +         * Set up 1:1 mapping for dom0. Default to include only
> +         * conventional RAM areas and let RMRRs include needed reserved
> +         * regions. When set, the inclusive mapping additionally maps in
> +         * every pfn up to 4GB except those that fall in unusable ranges.
> +         */
> +        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
> +            continue;
> +
> +        if ( iommu_dom0_inclusive && pfn <= max_pfn )
> +            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
> +        else
> +            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
> +
> +        if ( !map )
> +            continue;
> +
> +        /* Exclude Xen bits */
> +        if ( xen_in_range(pfn) )
> +            continue;
> +
> +        /*
> +         * If dom0-strict mode is enabled then exclude conventional RAM
> +         * and let the common code map dom0's pages.
> +         */
> +        if ( iommu_dom0_strict &&
> +             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
> +            continue;
> +
> +        rc = iommu_map_page(d, pfn, pfn,
> IOMMUF_readable|IOMMUF_writable);
> +        if ( rc )
> +            printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
> +                   d->domain_id, rc);
> +
> +        if (!(i & 0xfffff))
> +            process_pending_softirqs();
> +    }
> +}
> +
>  /*
>   * Local variables:
>   * mode: C
> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> index c0c6975ac4..99e5b89c0f 100644
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -37,6 +37,7 @@ extern bool_t iommu_debug;
>  extern bool_t amd_iommu_perdev_intremap;
> 
>  extern bool iommu_dom0_strict, iommu_dom0_passthrough;
> +extern int8_t iommu_dom0_inclusive;
> 
>  extern unsigned int iommu_dev_iotlb_timeout;
> 
> @@ -51,6 +52,7 @@ void arch_iommu_domain_destroy(struct domain *d);
>  int arch_iommu_domain_init(struct domain *d);
>  int arch_iommu_populate_page_table(struct domain *d);
>  void arch_iommu_check_autotranslated_hwdom(struct domain *d);
> +void arch_iommu_hwdom_init(struct domain *d);
> 
>  int iommu_construct(struct domain *d);
> 
> --
> 2.18.0
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
>
> Hello,
>
> The following series implement a workaround for missing RMRR
> entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> option.
>
> The PVH workaround identity maps all regions marked as reserved in the
> memory map.
>
> Note that this workaround is enabled by default on Intel hardware. It's
> also available to AMD hardware, although it's disabled by default in
> that case.
>
> The series can be found at:
>
> git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
>
> Thanks, Roger.
> Roger Pau Monne (4):
>   iommu: introduce dom0-iommu option
>   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
>   dom0/pvh: change the order of the MMCFG initialization
>   x86/iommu: add reserved dom0-iommu option to map reserved memory
>     ranges
>
>  docs/misc/xen-command-line.markdown         | 47 +++++++++++
>  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
>  xen/arch/x86/hvm/io.c                       |  5 ++
>  xen/arch/x86/x86_64/mm.c                    |  3 +-
>  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
>  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
>  xen/drivers/passthrough/arm/iommu.c         |  4 +
>  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
>  xen/drivers/passthrough/vtd/extern.h        |  2 -
>  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
>  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
>  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
>  xen/include/asm-x86/hvm/io.h                |  3 +
>  xen/include/xen/iommu.h                     |  8 +-
>  14 files changed, 240 insertions(+), 86 deletions(-)
>
> --

Hi Roger,
I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
4.18-rc8. I was able to get dom0 to boot with PVH but the physical
keyboard of the laptop stopped working, it works no problem with just
Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
running for a minute or two the system starts to slow down to the
point where it becomes unresponsive. The xl dmesg log is filled with
this error:

(XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
4625f3a000, iommu reg = ffff82c00181c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
(XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
(XEN)     root_entry[00] = 273a18001
(XEN)     context[10] = 2_27ba35001
(XEN)     l4[000] = 9c0000027ba34107
(XEN)     l3[118] = 8000000000000000
(XEN)     l3[118] not present

The device in question is:

00:02.0 VGA compatible controller: Intel Corporation HD Graphics 620
(rev 02) (prog-if 00 [VGA controller])
    Subsystem: Dell HD Graphics 620
    Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx+
    Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort-
<TAbort- <MAbort- >SERR- <PERR- INTx-
    Latency: 0
    Interrupt: pin A routed to IRQ 144
    Region 0: Memory at db000000 (64-bit, non-prefetchable) [size=16M]
    Region 2: Memory at 90000000 (64-bit, prefetchable) [size=256M]
    Region 4: I/O ports at f000 [size=64]
    [virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
    Capabilities: [40] Vendor Specific Information: Len=0c <?>
    Capabilities: [70] Express (v2) Root Complex Integrated Endpoint, MSI 00
        DevCap:    MaxPayload 128 bytes, PhantFunc 0
            ExtTag- RBE+
        DevCtl:    Report errors: Correctable- Non-Fatal- Fatal- Unsupported-
            RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-
            MaxPayload 128 bytes, MaxReadReq 128 bytes
        DevSta:    CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-
        DevCap2: Completion Timeout: Not Supported, TimeoutDis-, LTR-,
OBFF Not Supported
        DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-,
OBFF Disabled
    Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Address: fee00000  Data: 4028
    Capabilities: [d0] Power Management version 2
        Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA
PME(D0-,D1-,D2-,D3hot-,D3cold-)
        Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-
    Capabilities: [100 v1] Process Address Space ID (PASID)
        PASIDCap: Exec- Priv-, Max PASID Width: 14
        PASIDCtl: Enable- Exec- Priv-
    Capabilities: [200 v1] Address Translation Service (ATS)
        ATSCap:    Invalidate Queue Depth: 00
        ATSCtl:    Enable-, Smallest Translation Unit: 00
    Capabilities: [300 v1] Page Request Interface (PRI)
        PRICtl: Enable- Reset-
        PRISta: RF- UPRGI- Stopped+
        Page Request Capacity: 00008000, Page Request Allocation: 00000000
    Kernel driver in use: i915
    Kernel modules: i915

The error is similar to one I reported in 2015
(https://lists.xenproject.org/archives/html/xen-devel/2015-08/msg02339.html)
but that was due to AMT being enabled on the system, while this laptop
doesn't have AMT enabled. The boot params I had for Xen: loglvl=all
guest_loglvl=all dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null
dom0=pvh iommu=required,debug dom0-iommu=relaxed console=vga

Let us know if we can provide any additional information.

Thanks,
Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

[Paul Durrant]

> -----Original Message-----
> From: Xen-devel [mailto:xen-devel-bounces@lists.xenproject.org] On Behalf
> Of Roger Pau Monne
> Sent: 07 August 2018 15:03
> To: xen-devel@lists.xenproject.org
> Cc: Stefano Stabellini <sstabellini@kernel.org>; Wei Liu
> <wei.liu2@citrix.com>; George Dunlap <George.Dunlap@citrix.com>;
> Andrew Cooper <Andrew.Cooper3@citrix.com>; Ian Jackson
> <Ian.Jackson@citrix.com>; Tim (Xen.org) <tim@xen.org>; Julien Grall
> <julien.grall@arm.com>; Jan Beulich <jbeulich@suse.com>; Roger Pau
> Monne <roger.pau@citrix.com>
> Subject: [Xen-devel] [PATCH v3 4/4] x86/iommu: add reserved dom0-iommu
> option to map reserved memory ranges
> 
> Several people have reported hardware issues (malfunctioning USB
> controllers) due to iommu page faults on Intel hardware. Those faults
> are caused by missing RMRR (VTd) entries in the ACPI tables. Those can
> be worked around on VTd hardware by manually adding RMRR entries on
> the command line, this is however limited to Intel hardware and quite
> cumbersome to do.
> 
> In order to solve those issues add a new dom0-iommu=reserved option
> that identity maps all regions marked as reserved in the memory map.
> Note that regions used by devices emulated by Xen (LAPIC, IO-APIC or
> PCIe MCFG regions) are specifically avoided. Note that this option is
> available to a PVH Dom0 (as opposed to the inclusive option which only
> works for PV Dom0).
> 
> Signed-off-by: Roger Pau Monné <roger.pau@citrix.com>

Reviewed-by: Paul Durrant <paul.durrant@citrix.com>

> ---
> Changes since v2:
>  - Fix comment regarding dom0-strict.
>  - Change documentation style of xen command line.
>  - Rename iommu_map to hwdom_iommu_map.
>  - Move all the checks to hwdom_iommu_map.
> 
> Changes since v1:
>  - Introduce a new reserved option instead of abusing the inclusive
>    option.
>  - Use the same helper function for PV and PVH in order to decide if a
>    page should be added to the domain page tables.
>  - Use the data inside of the domain struct to detect overlaps with
>    emulated MMIO regions.
> ---
> Cc: Andrew Cooper <andrew.cooper3@citrix.com>
> Cc: George Dunlap <George.Dunlap@eu.citrix.com>
> Cc: Ian Jackson <ian.jackson@eu.citrix.com>
> Cc: Jan Beulich <jbeulich@suse.com>
> Cc: Julien Grall <julien.grall@arm.com>
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: Stefano Stabellini <sstabellini@kernel.org>
> Cc: Tim Deegan <tim@xen.org>
> Cc: Wei Liu <wei.liu2@citrix.com>
> ---
>  docs/misc/xen-command-line.markdown         | 11 ++-
>  xen/arch/x86/hvm/io.c                       |  5 ++
>  xen/drivers/passthrough/amd/pci_amd_iommu.c |  3 +
>  xen/drivers/passthrough/iommu.c             |  3 +
>  xen/drivers/passthrough/vtd/iommu.c         |  3 +
>  xen/drivers/passthrough/x86/iommu.c         | 86 ++++++++++++++-------
>  xen/include/asm-x86/hvm/io.h                |  3 +
>  xen/include/xen/iommu.h                     |  2 +-
>  8 files changed, 85 insertions(+), 31 deletions(-)
> 
> diff --git a/docs/misc/xen-command-line.markdown b/docs/misc/xen-
> command-line.markdown
> index 90b32fe3f0..59ec2afc5d 100644
> --- a/docs/misc/xen-command-line.markdown
> +++ b/docs/misc/xen-command-line.markdown
> @@ -1205,7 +1205,7 @@ detection of systems known to misbehave upon
> accesses to that port.
>  >> Enable IOMMU debugging code (implies `verbose`).
> 
>  ### dom0-iommu
> -> `= List of [ none | strict | relaxed | inclusive ]`
> +> `= List of [ none | strict | relaxed | inclusive | reserved ]`
> 
>  * `none`: disables DMA remapping for Dom0.
> 
> @@ -1233,6 +1233,15 @@ meaning:
>    option is only applicable to a PV Dom0 and is enabled by default on Intel
>    hardware.
> 
> +* `reserved`: sets up DMA remapping for all the reserved regions in the
> memory
> +  map for Dom0. Use this to work around firmware issues providing incorrect
> +  RMRR/IVMD entries. Rather than only mapping RAM pages for IOMMU
> accesses
> +  for Dom0, all memory regions marked as reserved in the memory map that
> don't
> +  overlap with any MMIO region from emulated devices will be identity
> mapped.
> +  This option maps a subset of the memory that would be mapped when
> using the
> +  `inclusive` option. This option is available to a PVH Dom0 and is enabled by
> +  default on Intel hardware.
> +
>  ### iommu\_dev\_iotlb\_timeout
>  > `= <integer>`
> 
> diff --git a/xen/arch/x86/hvm/io.c b/xen/arch/x86/hvm/io.c
> index bf4d8748d3..5e01c33890 100644
> --- a/xen/arch/x86/hvm/io.c
> +++ b/xen/arch/x86/hvm/io.c
> @@ -404,6 +404,11 @@ static const struct hvm_mmcfg
> *vpci_mmcfg_find(const struct domain *d,
>      return NULL;
>  }
> 
> +bool vpci_mmcfg_address(const struct domain *d, paddr_t addr)
> +{
> +    return vpci_mmcfg_find(d, addr);
> +}
> +
>  static unsigned int vpci_mmcfg_decode_addr(const struct hvm_mmcfg
> *mmcfg,
>                                             paddr_t addr, pci_sbdf_t *sbdf)
>  {
> diff --git a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> index 0e0c99c942..2c2867d088 100644
> --- a/xen/drivers/passthrough/amd/pci_amd_iommu.c
> +++ b/xen/drivers/passthrough/amd/pci_amd_iommu.c
> @@ -256,6 +256,9 @@ static void __hwdom_init
> amd_iommu_hwdom_init(struct domain *d)
>      /* Inclusive IOMMU mappings are disabled by default on AMD hardware.
> */
>      iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ? false
>                                                        : iommu_dom0_inclusive;
> +    /* Reserved IOMMU mappings are disabled by default on AMD
> hardware. */
> +    iommu_dom0_reserved = iommu_dom0_reserved == -1 ? false
> +                                                    : iommu_dom0_reserved;
> 
>      if ( allocate_domain_resources(dom_iommu(d)) )
>          BUG();
> diff --git a/xen/drivers/passthrough/iommu.c
> b/xen/drivers/passthrough/iommu.c
> index f15c94be42..9c991bd2cf 100644
> --- a/xen/drivers/passthrough/iommu.c
> +++ b/xen/drivers/passthrough/iommu.c
> @@ -75,6 +75,7 @@ custom_param("dom0-iommu",
> parse_dom0_iommu_param);
>  bool __hwdom_initdata iommu_dom0_strict;
>  bool __read_mostly iommu_dom0_passthrough;
>  int8_t __hwdom_initdata iommu_dom0_inclusive = -1;
> +int8_t __hwdom_initdata iommu_dom0_reserved = -1;
> 
>  DEFINE_PER_CPU(bool_t, iommu_dont_flush_iotlb);
> 
> @@ -162,6 +163,8 @@ static int __init parse_dom0_iommu_param(const
> char *s)
>              iommu_dom0_strict = false;
>          else if ( !strncmp(s, "inclusive", ss - s) )
>              iommu_dom0_inclusive = val;
> +        else if ( !strncmp(s, "reserved", ss - s) )
> +            iommu_dom0_reserved = val;
>          else
>              rc = -EINVAL;
> 
> diff --git a/xen/drivers/passthrough/vtd/iommu.c
> b/xen/drivers/passthrough/vtd/iommu.c
> index 7c7e15755d..77a076215b 100644
> --- a/xen/drivers/passthrough/vtd/iommu.c
> +++ b/xen/drivers/passthrough/vtd/iommu.c
> @@ -1307,6 +1307,9 @@ static void __hwdom_init
> intel_iommu_hwdom_init(struct domain *d)
>      /* Inclusive mappings are enabled by default on Intel hardware for PV. */
>      iommu_dom0_inclusive = iommu_dom0_inclusive == -1 ?
> is_pv_domain(d)
>                                                        : iommu_dom0_inclusive;
> +    /* Reserved IOMMU mappings are enabled by default on Intel hardware.
> */
> +    iommu_dom0_reserved = iommu_dom0_reserved == -1 ? true
> +                                                    : iommu_dom0_reserved;
> 
>      setup_hwdom_pci_devices(d, setup_hwdom_device);
>      setup_hwdom_rmrr(d);
> diff --git a/xen/drivers/passthrough/x86/iommu.c
> b/xen/drivers/passthrough/x86/iommu.c
> index 5a7a765e9d..6aec43ed1a 100644
> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -20,6 +20,7 @@
>  #include <xen/softirq.h>
>  #include <xsm/xsm.h>
> 
> +#include <asm/hvm/io.h>
>  #include <asm/setup.h>
> 
>  void iommu_update_ire_from_apic(
> @@ -134,13 +135,67 @@ void arch_iommu_domain_destroy(struct domain
> *d)
>  {
>  }
> 
> +static bool __hwdom_init hwdom_iommu_map(const struct domain *d,
> unsigned long pfn,
> +                                         unsigned long max_pfn)
> +{
> +    unsigned int i;
> +
> +    /*
> +     * Ignore any address below 1MB, that's already identity mapped by the
> +     * domain builder for HVM.
> +     */
> +    if ( (is_hvm_domain(d) && pfn < PFN_DOWN(MB(1))) ||
> +         /* Exclude Xen bits. */
> +         xen_in_range(pfn) || (pfn > max_pfn && !mfn_valid(_mfn(pfn))) )
> +        return false;
> +
> +    /*
> +     * If dom0-strict mode is enabled or the guest type is PVH/HVM then
> exclude
> +     * conventional RAM and let the common code map dom0's pages.
> +     */
> +    if ( page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
> +         (iommu_dom0_strict || is_hvm_domain(d)) )
> +        return false;
> +    if ( page_is_ram_type(pfn, RAM_TYPE_RESERVED) &&
> +         !iommu_dom0_reserved && !iommu_dom0_inclusive )
> +        return false;
> +    if ( !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE) &&
> +         !page_is_ram_type(pfn, RAM_TYPE_RESERVED) &&
> +         !page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) &&
> +         (!iommu_dom0_inclusive || pfn > max_pfn) )
> +        return false;
> +
> +    /* Check that it doesn't overlap with the LAPIC */
> +    if ( has_vlapic(d) )
> +    {
> +        const struct vcpu *v;
> +
> +        for_each_vcpu(d, v)
> +            if ( pfn == PFN_DOWN(vlapic_base_address(vcpu_vlapic(v))) )
> +                return false;
> +    }
> +    /* ... or the IO-APIC */
> +    for ( i = 0; has_vioapic(d) && i < d->arch.hvm_domain.nr_vioapics; i++ )
> +        if ( pfn == PFN_DOWN(domain_vioapic(d, i)->base_address) )
> +            return false;
> +    /*
> +     * ... or the PCIe MCFG regions.
> +     * TODO: runtime added MMCFG regions are not checked to make sure
> they
> +     * don't overlap with already mapped regions, thus preventing trapping.
> +     */
> +    if ( has_vpci(d) && vpci_mmcfg_address(d, pfn << PAGE_SHIFT) )
> +        return false;
> +
> +    return true;
> +}
> +
>  void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>  {
>      unsigned long i, top, max_pfn;
> 
>      BUG_ON(!is_hardware_domain(d));
> 
> -    if ( iommu_dom0_passthrough || !is_pv_domain(d) )
> +    if ( iommu_dom0_passthrough )
>          return;
> 
>      max_pfn = (GB(4) >> PAGE_SHIFT) - 1;
> @@ -149,36 +204,9 @@ void __hwdom_init arch_iommu_hwdom_init(struct
> domain *d)
>      for ( i = 0; i < top; i++ )
>      {
>          unsigned long pfn = pdx_to_pfn(i);
> -        bool map;
>          int rc;
> 
> -        /*
> -         * Set up 1:1 mapping for dom0. Default to include only
> -         * conventional RAM areas and let RMRRs include needed reserved
> -         * regions. When set, the inclusive mapping additionally maps in
> -         * every pfn up to 4GB except those that fall in unusable ranges.
> -         */
> -        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
> -            continue;
> -
> -        if ( iommu_dom0_inclusive && pfn <= max_pfn )
> -            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
> -        else
> -            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
> -
> -        if ( !map )
> -            continue;
> -
> -        /* Exclude Xen bits */
> -        if ( xen_in_range(pfn) )
> -            continue;
> -
> -        /*
> -         * If dom0-strict mode is enabled then exclude conventional RAM
> -         * and let the common code map dom0's pages.
> -         */
> -        if ( iommu_dom0_strict &&
> -             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
> +        if ( !hwdom_iommu_map(d, pfn, max_pfn) )
>              continue;
> 
>          rc = iommu_map_page(d, pfn, pfn,
> IOMMUF_readable|IOMMUF_writable);
> diff --git a/xen/include/asm-x86/hvm/io.h b/xen/include/asm-x86/hvm/io.h
> index e6b6ed0b92..8cca456b55 100644
> --- a/xen/include/asm-x86/hvm/io.h
> +++ b/xen/include/asm-x86/hvm/io.h
> @@ -180,6 +180,9 @@ int register_vpci_mmcfg_handler(struct domain *d,
> paddr_t addr,
>  /* Destroy tracked MMCFG areas. */
>  void destroy_vpci_mmcfg(struct domain *d);
> 
> +/* Check if an address is between a MMCFG region for a domain. */
> +bool vpci_mmcfg_address(const struct domain *d, paddr_t addr);
> +
>  #endif /* __ASM_X86_HVM_IO_H__ */
> 
> 
> diff --git a/xen/include/xen/iommu.h b/xen/include/xen/iommu.h
> index 99e5b89c0f..fed1b1ea7a 100644
> --- a/xen/include/xen/iommu.h
> +++ b/xen/include/xen/iommu.h
> @@ -37,7 +37,7 @@ extern bool_t iommu_debug;
>  extern bool_t amd_iommu_perdev_intremap;
> 
>  extern bool iommu_dom0_strict, iommu_dom0_passthrough;
> -extern int8_t iommu_dom0_inclusive;
> +extern int8_t iommu_dom0_inclusive, iommu_dom0_reserved;
> 
>  extern unsigned int iommu_dev_iotlb_timeout;
> 
> --
> 2.18.0
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xenproject.org
> https://lists.xenproject.org/mailman/listinfo/xen-devel
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Roger Pau Monné]

On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
> On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> >
> > Hello,
> >
> > The following series implement a workaround for missing RMRR
> > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> > option.
> >
> > The PVH workaround identity maps all regions marked as reserved in the
> > memory map.
> >
> > Note that this workaround is enabled by default on Intel hardware. It's
> > also available to AMD hardware, although it's disabled by default in
> > that case.
> >
> > The series can be found at:
> >
> > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
> >
> > Thanks, Roger.
> > Roger Pau Monne (4):
> >   iommu: introduce dom0-iommu option
> >   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
> >   dom0/pvh: change the order of the MMCFG initialization
> >   x86/iommu: add reserved dom0-iommu option to map reserved memory
> >     ranges
> >
> >  docs/misc/xen-command-line.markdown         | 47 +++++++++++
> >  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
> >  xen/arch/x86/hvm/io.c                       |  5 ++
> >  xen/arch/x86/x86_64/mm.c                    |  3 +-
> >  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
> >  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
> >  xen/drivers/passthrough/arm/iommu.c         |  4 +
> >  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
> >  xen/drivers/passthrough/vtd/extern.h        |  2 -
> >  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
> >  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
> >  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
> >  xen/include/asm-x86/hvm/io.h                |  3 +
> >  xen/include/xen/iommu.h                     |  8 +-
> >  14 files changed, 240 insertions(+), 86 deletions(-)
> >
> > --
> 
> Hi Roger,
> I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
> 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
> keyboard of the laptop stopped working, it works no problem with just
> Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
> running for a minute or two the system starts to slow down to the
> point where it becomes unresponsive. The xl dmesg log is filled with
> this error:
> 
> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> 4625f3a000, iommu reg = ffff82c00181c000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a

Is the gmfn always the same (0x4625f3a)?

> (XEN)     root_entry[00] = 273a18001
> (XEN)     context[10] = 2_27ba35001
> (XEN)     l4[000] = 9c0000027ba34107
> (XEN)     l3[118] = 8000000000000000
> (XEN)     l3[118] not present

Can you also paste the full xl dmesg log? I'm specially interested in
the memory map of the machine which is printed quite early during Xen
boot.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
>
> On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
> > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> > >
> > > Hello,
> > >
> > > The following series implement a workaround for missing RMRR
> > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> > > option.
> > >
> > > The PVH workaround identity maps all regions marked as reserved in the
> > > memory map.
> > >
> > > Note that this workaround is enabled by default on Intel hardware. It's
> > > also available to AMD hardware, although it's disabled by default in
> > > that case.
> > >
> > > The series can be found at:
> > >
> > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
> > >
> > > Thanks, Roger.
> > > Roger Pau Monne (4):
> > >   iommu: introduce dom0-iommu option
> > >   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
> > >   dom0/pvh: change the order of the MMCFG initialization
> > >   x86/iommu: add reserved dom0-iommu option to map reserved memory
> > >     ranges
> > >
> > >  docs/misc/xen-command-line.markdown         | 47 +++++++++++
> > >  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
> > >  xen/arch/x86/hvm/io.c                       |  5 ++
> > >  xen/arch/x86/x86_64/mm.c                    |  3 +-
> > >  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
> > >  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
> > >  xen/drivers/passthrough/arm/iommu.c         |  4 +
> > >  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
> > >  xen/drivers/passthrough/vtd/extern.h        |  2 -
> > >  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
> > >  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
> > >  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
> > >  xen/include/asm-x86/hvm/io.h                |  3 +
> > >  xen/include/xen/iommu.h                     |  8 +-
> > >  14 files changed, 240 insertions(+), 86 deletions(-)
> > >
> > > --
> >
> > Hi Roger,
> > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
> > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
> > keyboard of the laptop stopped working, it works no problem with just
> > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
> > running for a minute or two the system starts to slow down to the
> > point where it becomes unresponsive. The xl dmesg log is filled with
> > this error:
> >
> > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> > 4625f3a000, iommu reg = ffff82c00181c000
> > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> > (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
>
> Is the gmfn always the same (0x4625f3a)?
>
> > (XEN)     root_entry[00] = 273a18001
> > (XEN)     context[10] = 2_27ba35001
> > (XEN)     l4[000] = 9c0000027ba34107
> > (XEN)     l3[118] = 8000000000000000
> > (XEN)     l3[118] not present
>
> Can you also paste the full xl dmesg log? I'm specially interested in
> the memory map of the machine which is printed quite early during Xen
> boot.
>

Unfortunately I don't have serial access on this laptop and "xl dmesg"
gets completely filled with that error so the beginning of the log is
lost by the time I get a terminal in dom0. I'll try it on another
laptop later today for which I have serial access. Across reboots the
gmfn changes:

(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
23c251000, iommu reg = ffff82c00181c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
(XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 23c251
(XEN)     root_entry[00] = 273a18001
(XEN)     context[10] = 2_27ba35001
(XEN)     l4[000] = 9c0000027ba34107
(XEN)     l3[008] = 8000000000000000
(XEN)     l3[008] not present

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Roger Pau Monné]

On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote:
> On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> >
> > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
> > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > The following series implement a workaround for missing RMRR
> > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> > > > option.
> > > >
> > > > The PVH workaround identity maps all regions marked as reserved in the
> > > > memory map.
> > > >
> > > > Note that this workaround is enabled by default on Intel hardware. It's
> > > > also available to AMD hardware, although it's disabled by default in
> > > > that case.
> > > >
> > > > The series can be found at:
> > > >
> > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
> > > >
> > > > Thanks, Roger.
> > > > Roger Pau Monne (4):
> > > >   iommu: introduce dom0-iommu option
> > > >   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
> > > >   dom0/pvh: change the order of the MMCFG initialization
> > > >   x86/iommu: add reserved dom0-iommu option to map reserved memory
> > > >     ranges
> > > >
> > > >  docs/misc/xen-command-line.markdown         | 47 +++++++++++
> > > >  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
> > > >  xen/arch/x86/hvm/io.c                       |  5 ++
> > > >  xen/arch/x86/x86_64/mm.c                    |  3 +-
> > > >  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
> > > >  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
> > > >  xen/drivers/passthrough/arm/iommu.c         |  4 +
> > > >  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
> > > >  xen/drivers/passthrough/vtd/extern.h        |  2 -
> > > >  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
> > > >  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
> > > >  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
> > > >  xen/include/asm-x86/hvm/io.h                |  3 +
> > > >  xen/include/xen/iommu.h                     |  8 +-
> > > >  14 files changed, 240 insertions(+), 86 deletions(-)
> > > >
> > > > --
> > >
> > > Hi Roger,
> > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
> > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
> > > keyboard of the laptop stopped working, it works no problem with just
> > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
> > > running for a minute or two the system starts to slow down to the
> > > point where it becomes unresponsive. The xl dmesg log is filled with
> > > this error:
> > >
> > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> > > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> > > 4625f3a000, iommu reg = ffff82c00181c000
> > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> > > (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
> >
> > Is the gmfn always the same (0x4625f3a)?
> >
> > > (XEN)     root_entry[00] = 273a18001
> > > (XEN)     context[10] = 2_27ba35001
> > > (XEN)     l4[000] = 9c0000027ba34107
> > > (XEN)     l3[118] = 8000000000000000
> > > (XEN)     l3[118] not present
> >
> > Can you also paste the full xl dmesg log? I'm specially interested in
> > the memory map of the machine which is printed quite early during Xen
> > boot.
> >
> 
> Unfortunately I don't have serial access on this laptop and "xl dmesg"
> gets completely filled with that error so the beginning of the log is
> lost by the time I get a terminal in dom0.

You can get the memory map while booting in PV mode, it's going to be
exactly the same regardless of whether Dom0 is PV or PVH.

> I'll try it on another
> laptop later today for which I have serial access. Across reboots the
> gmfn changes:
> 
> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> 23c251000, iommu reg = ffff82c00181c000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 23c251
> (XEN)     root_entry[00] = 273a18001
> (XEN)     context[10] = 2_27ba35001
> (XEN)     l4[000] = 9c0000027ba34107
> (XEN)     l3[008] = 8000000000000000
> (XEN)     l3[008] not present
> 

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Andrew Cooper]

On 07/08/18 15:45, Tamas K Lengyel wrote:
> On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
>> On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
>>> On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
>>>> Hello,
>>>>
>>>> The following series implement a workaround for missing RMRR
>>>> entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
>>>> option.
>>>>
>>>> The PVH workaround identity maps all regions marked as reserved in the
>>>> memory map.
>>>>
>>>> Note that this workaround is enabled by default on Intel hardware. It's
>>>> also available to AMD hardware, although it's disabled by default in
>>>> that case.
>>>>
>>>> The series can be found at:
>>>>
>>>> git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
>>>>
>>>> Thanks, Roger.
>>>> Roger Pau Monne (4):
>>>>   iommu: introduce dom0-iommu option
>>>>   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
>>>>   dom0/pvh: change the order of the MMCFG initialization
>>>>   x86/iommu: add reserved dom0-iommu option to map reserved memory
>>>>     ranges
>>>>
>>>>  docs/misc/xen-command-line.markdown         | 47 +++++++++++
>>>>  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
>>>>  xen/arch/x86/hvm/io.c                       |  5 ++
>>>>  xen/arch/x86/x86_64/mm.c                    |  3 +-
>>>>  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
>>>>  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
>>>>  xen/drivers/passthrough/arm/iommu.c         |  4 +
>>>>  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
>>>>  xen/drivers/passthrough/vtd/extern.h        |  2 -
>>>>  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
>>>>  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
>>>>  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
>>>>  xen/include/asm-x86/hvm/io.h                |  3 +
>>>>  xen/include/xen/iommu.h                     |  8 +-
>>>>  14 files changed, 240 insertions(+), 86 deletions(-)
>>>>
>>>> --
>>> Hi Roger,
>>> I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
>>> 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
>>> keyboard of the laptop stopped working, it works no problem with just
>>> Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
>>> running for a minute or two the system starts to slow down to the
>>> point where it becomes unresponsive. The xl dmesg log is filled with
>>> this error:
>>>
>>> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
>>> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
>>> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
>>> 4625f3a000, iommu reg = ffff82c00181c000
>>> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>>> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
>> Is the gmfn always the same (0x4625f3a)?
>>
>>> (XEN)     root_entry[00] = 273a18001
>>> (XEN)     context[10] = 2_27ba35001
>>> (XEN)     l4[000] = 9c0000027ba34107
>>> (XEN)     l3[118] = 8000000000000000
>>> (XEN)     l3[118] not present
>> Can you also paste the full xl dmesg log? I'm specially interested in
>> the memory map of the machine which is printed quite early during Xen
>> boot.
>>
> Unfortunately I don't have serial access on this laptop and "xl dmesg"
> gets completely filled with that error so the beginning of the log is
> lost by the time I get a terminal in dom0.

diff --git a/xen/drivers/char/console.c b/xen/drivers/char/console.c
index a911958..9c1ee0a 100644
--- a/xen/drivers/char/console.c
+++ b/xen/drivers/char/console.c
@@ -81,7 +81,7 @@ custom_runtime_param("console_timestamps",
parse_console_timestamps);
 static uint32_t __initdata opt_conring_size;
 size_param("conring_size", opt_conring_size);
 
-#define _CONRING_SIZE 16384
+#define _CONRING_SIZE KB(512)
 #define CONRING_IDX_MASK(i) ((i)&(conring_size-1))
 static char __initdata _conring[_CONRING_SIZE];
 static char *__read_mostly conring = _conring;

Sadly, the conring_size= option is almost completely useless, because
the smaller ring tends to truncate before it gets realloc()'d to be larger.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Tue, Aug 7, 2018 at 9:09 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
>
> On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote:
> > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> > >
> > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
> > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> > > > >
> > > > > Hello,
> > > > >
> > > > > The following series implement a workaround for missing RMRR
> > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> > > > > option.
> > > > >
> > > > > The PVH workaround identity maps all regions marked as reserved in the
> > > > > memory map.
> > > > >
> > > > > Note that this workaround is enabled by default on Intel hardware. It's
> > > > > also available to AMD hardware, although it's disabled by default in
> > > > > that case.
> > > > >
> > > > > The series can be found at:
> > > > >
> > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
> > > > >
> > > > > Thanks, Roger.
> > > > > Roger Pau Monne (4):
> > > > >   iommu: introduce dom0-iommu option
> > > > >   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
> > > > >   dom0/pvh: change the order of the MMCFG initialization
> > > > >   x86/iommu: add reserved dom0-iommu option to map reserved memory
> > > > >     ranges
> > > > >
> > > > >  docs/misc/xen-command-line.markdown         | 47 +++++++++++
> > > > >  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
> > > > >  xen/arch/x86/hvm/io.c                       |  5 ++
> > > > >  xen/arch/x86/x86_64/mm.c                    |  3 +-
> > > > >  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
> > > > >  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
> > > > >  xen/drivers/passthrough/arm/iommu.c         |  4 +
> > > > >  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
> > > > >  xen/drivers/passthrough/vtd/extern.h        |  2 -
> > > > >  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
> > > > >  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
> > > > >  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
> > > > >  xen/include/asm-x86/hvm/io.h                |  3 +
> > > > >  xen/include/xen/iommu.h                     |  8 +-
> > > > >  14 files changed, 240 insertions(+), 86 deletions(-)
> > > > >
> > > > > --
> > > >
> > > > Hi Roger,
> > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
> > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
> > > > keyboard of the laptop stopped working, it works no problem with just
> > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
> > > > running for a minute or two the system starts to slow down to the
> > > > point where it becomes unresponsive. The xl dmesg log is filled with
> > > > this error:
> > > >
> > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> > > > 4625f3a000, iommu reg = ffff82c00181c000
> > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> > > > (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
> > >
> > > Is the gmfn always the same (0x4625f3a)?
> > >
> > > > (XEN)     root_entry[00] = 273a18001
> > > > (XEN)     context[10] = 2_27ba35001
> > > > (XEN)     l4[000] = 9c0000027ba34107
> > > > (XEN)     l3[118] = 8000000000000000
> > > > (XEN)     l3[118] not present
> > >
> > > Can you also paste the full xl dmesg log? I'm specially interested in
> > > the memory map of the machine which is printed quite early during Xen
> > > boot.
> > >
> >
> > Unfortunately I don't have serial access on this laptop and "xl dmesg"
> > gets completely filled with that error so the beginning of the log is
> > lost by the time I get a terminal in dom0.
>
> You can get the memory map while booting in PV mode, it's going to be
> exactly the same regardless of whether Dom0 is PV or PVH.

This is the PV dmesg:

(XEN) Xen version 4.12-unstable (dr@) (gcc (Debian 7.3.0-19) 7.3.0)
debug=y  Mon Aug  6 13:42:42 MDT 2018
(XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f
(XEN) Bootloader: EFI
(XEN) Command line: loglvl=all guest_loglvl=all
dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null console=vga
(XEN) Xen image load base address: 0x5a200000
(XEN) Video information:
(XEN)  VGA is graphics mode 3200x1800, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 0000000000058000 (usable)
(XEN)  0000000000058000 - 0000000000059000 (reserved)
(XEN)  0000000000059000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 000000005f14d000 (usable)
(XEN)  000000005f14d000 - 000000005f14e000 (ACPI NVS)
(XEN)  000000005f14e000 - 000000005f14f000 (reserved)
(XEN)  000000005f14f000 - 000000006ee89000 (usable)
(XEN)  000000006ee89000 - 000000006f214000 (reserved)
(XEN)  000000006f214000 - 000000006f258000 (ACPI data)
(XEN)  000000006f258000 - 000000006f8fd000 (ACPI NVS)
(XEN)  000000006f8fd000 - 000000006ffff000 (reserved)
(XEN)  000000006ffff000 - 0000000070000000 (usable)
(XEN)  0000000070000000 - 0000000078000000 (reserved)
(XEN)  0000000078000000 - 0000000078600000 (usable)
(XEN)  0000000078600000 - 000000007c800000 (reserved)
(XEN)  00000000e0000000 - 00000000f0000000 (reserved)
(XEN)  00000000fe000000 - 00000000fe011000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 0000000281800000 (usable)
(XEN) ACPI: RSDP 6F221000, 0024 (r2 DELL  )
(XEN) ACPI: XSDT 6F2210C8, 0114 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FACP 6F246918, 010C (r5 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: DSDT 6F221270, 256A6 (r2 DELL    CBX3     1072009 INTL 20160422)
(XEN) ACPI: FACS 6F8FA180, 0040
(XEN) ACPI: APIC 6F246A28, 0084 (r3 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FPDT 6F246AB0, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FIDT 6F246AF8, 00AC (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: MCFG 6F246BA8, 003C (r1 DELL    CBX3     1072009 MSFT       97)
(XEN) ACPI: HPET 6F246BE8, 0038 (r1 DELL    CBX3     1072009 AMI.    5000B)
(XEN) ACPI: SSDT 6F246C20, 0359 (r1 SataRe SataTabl     1000 INTL 20160422)
(XEN) ACPI: BOOT 6F246F80, 0028 (r1 DELL   CBX3      1072009 AMI     10013)
(XEN) ACPI: SSDT 6F246FA8, 12CF (r2 SaSsdt  SaSsdt      3000 INTL 20160422)
(XEN) ACPI: HPET 6F248278, 0038 (r1 INTEL   KBL-ULT        1 MSFT       5F)
(XEN) ACPI: SSDT 6F2482B0, 0D84 (r2  INTEL xh_rvp07        0 INTL 20160422)
(XEN) ACPI: UEFI 6F249038, 0042 (r1                        0             0)
(XEN) ACPI: SSDT 6F249080, 0EDE (r2 CpuRef  CpuSsdt     3000 INTL 20160422)
(XEN) ACPI: LPIT 6F249F60, 0094 (r1 INTEL   KBL-ULT        0 MSFT       5F)
(XEN) ACPI: WSMT 6F249FF8, 0028 (r1 DELL    CBX3           0 MSFT       5F)
(XEN) ACPI: SSDT 6F24A020, 0161 (r2 INTEL    HdaDsp        0 INTL 20160422)
(XEN) ACPI: SSDT 6F24A188, 029F (r2 INTEL  sensrhub        0 INTL 20160422)
(XEN) ACPI: SSDT 6F24A428, 3002 (r2 INTEL  PtidDevc     1000 INTL 20160422)
(XEN) ACPI: SSDT 6F24D430, 00DB (r2 INTEL  TbtTypeC        0 INTL 20160422)
(XEN) ACPI: DBGP 6F24D510, 0034 (r1 INTEL                  2 MSFT       5F)
(XEN) ACPI: DBG2 6F24D548, 0054 (r0 INTEL                  2 MSFT       5F)
(XEN) ACPI: SSDT 6F24D5A0, 07AC (r2 INTEL  UsbCTabl     1000 INTL 20160422)
(XEN) ACPI: SSDT 6F24DD50, 84F1 (r2 DptfTa DptfTabl     1000 INTL 20160422)
(XEN) ACPI: MSDM 6F256248, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
(XEN) ACPI: SLIC 6F2562A0, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
(XEN) ACPI: NHLT 6F256418, 002D (r0 INTEL  EDK2            2       1000013)
(XEN) ACPI: BGRT 6F256448, 0038 (r0     ��           1072009 AMI     10013)
(XEN) ACPI: TPM2 6F256480, 0034 (r3        Tpm2Tabl        1 AMI         0)
(XEN) ACPI: ASF! 6F2564B8, 00A0 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) ACPI: DMAR 6F256558, 00F0 (r1 INTEL      KBL         1 INTL        1)
(XEN) System RAM: 7948MB (8138904kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-0000000281800000
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0x90000000, mapped to 0xffff82c000201000,
using 22528k, total 22528k
(XEN) vesafb: mode is 3200x1800x32, linelength=12800, font 8x16
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) Couldn't initialize a 3200x1800 framebuffer early.
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 142 (0x8e), Stepping 9
(raw 000806e9)
(XEN) SMBIOS 3.0 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1808 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
(XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
6f8fa180/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[6f8fa18c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]dmar.c:851: Ignore unknown DMAR structure type (0x4)
(XEN) [VT-D]dmar.c:851: Ignore unknown DMAR structure type (0x4)
(XEN) ERST table was not found
(XEN) ACPI: BGRT: invalidating v1 image at 0x6ad04018
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) mce_intel.c:780: MCA Capability: firstbank 0, extended MCE MSR
0, BCAST, CMCI
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP
(XEN)   Compiled-in support: INDIRECT_THUNK
(XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+, Other: IBPB
(XEN)   Support for VMs: PV: MSR_SPEC_CTRL RSB EAGER_FPU, HVM:
MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled
(XEN) Using scheduler: null Scheduler (null)
(XEN) Initializing null scheduler
(XEN) WARNING: This is experimental software in development.
(XEN) Use at your own risk.
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 2904.041 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
(XEN)  0000000008000-0000000057fff type=7 attr=000000000000000f
(XEN)  0000000058000-0000000058fff type=0 attr=000000000000000f
(XEN)  0000000059000-000000005cfff type=7 attr=000000000000000f
(XEN)  000000005d000-000000005efff type=2 attr=000000000000000f
(XEN)  000000005f000-000000005ffff type=4 attr=000000000000000f
(XEN)  0000000060000-000000009efff type=3 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-000003f225fff type=7 attr=000000000000000f
(XEN)  000003f226000-000003f265fff type=4 attr=000000000000000f
(XEN)  000003f266000-00000561bbfff type=7 attr=000000000000000f
(XEN)  00000561bc000-000005a03afff type=2 attr=000000000000000f
(XEN)  000005a03b000-000005b23afff type=1 attr=000000000000000f
(XEN)  000005b23b000-000005b3e5fff type=2 attr=000000000000000f
(XEN)  000005b3e6000-000005b3e9fff type=7 attr=000000000000000f
(XEN)  000005b3ea000-000005b415fff type=2 attr=000000000000000f
(XEN)  000005b416000-000005b420fff type=7 attr=000000000000000f
(XEN)  000005b421000-000005b458fff type=2 attr=000000000000000f
(XEN)  000005b459000-000005b45afff type=7 attr=000000000000000f
(XEN)  000005b45b000-000005b462fff type=2 attr=000000000000000f
(XEN)  000005b463000-000005b464fff type=7 attr=000000000000000f
(XEN)  000005b465000-000005d2e0fff type=2 attr=000000000000000f
(XEN)  000005d2e1000-000005d325fff type=1 attr=000000000000000f
(XEN)  000005d326000-000005f14cfff type=4 attr=000000000000000f
(XEN)  000005f14d000-000005f14dfff type=10 attr=000000000000000f
(XEN)  000005f14e000-000005f14efff type=6 attr=800000000000000f
(XEN)  000005f14f000-000005f215fff type=4 attr=000000000000000f
(XEN)  000005f216000-000005f224fff type=2 attr=000000000000000f
(XEN)  000005f225000-000006dfe6fff type=4 attr=000000000000000f
(XEN)  000006dfe7000-000006e2fafff type=7 attr=000000000000000f
(XEN)  000006e2fb000-000006ee88fff type=3 attr=000000000000000f
(XEN)  000006ee89000-000006f213fff type=0 attr=000000000000000f
(XEN)  000006f214000-000006f257fff type=9 attr=000000000000000f
(XEN)  000006f258000-000006f8fcfff type=10 attr=000000000000000f
(XEN)  000006f8fd000-000006ff1ffff type=6 attr=800000000000000f
(XEN)  000006ff20000-000006fffefff type=5 attr=800000000000000f
(XEN)  000006ffff000-000006fffffff type=4 attr=000000000000000f
(XEN)  0000078000000-00000785fffff type=7 attr=000000000000000f
(XEN)  0000100000000-00002817fffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  0000070000000-0000077ffffff type=0 attr=070000000000000f
(XEN)  0000078600000-000007c7fffff type=0 attr=0000000000000000
(XEN)  00000e0000000-00000efffffff type=11 attr=8000000000000000
(XEN) Unknown cachability for MFNs 0xe0000-0xeffff
(XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
(XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
(XEN)  00000fee00000-00000fee00fff type=11 attr=800000000000100d
(XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0806739b0 -> ffff82d0806757c4
(XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
(XEN) PCI: MCFG area at e0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-ff
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=0 pin2=0
(XEN) TSC deadline timer enabled
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x11142120
(XEN) mwait-idle: v0.4.1 model 0x8e
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Brought up 4 CPUs
(XEN) build-id: 1e3eb01a83018530b8a66809744145e8
(XEN) Running stub recovery selftests...
(XEN) traps.c:1570: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] ->
ffff82d08037f3f2
(XEN) traps.c:755: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] ->
ffff82d08037f3f2
(XEN) traps.c:1097: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] ->
ffff82d08037f3f2
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 504 PIRQs
(XEN) sched_null.c:347: 0 <-- d0v0
(XEN) NX (Execute Disable) protection active
(XEN) *** Building a PV Dom0 ***
(XEN) ELF: phdr: paddr=0x1000000 memsz=0x123c000
(XEN) ELF: phdr: paddr=0x2400000 memsz=0x2a7000
(XEN) ELF: phdr: paddr=0x26a7000 memsz=0x25000
(XEN) ELF: phdr: paddr=0x26cc000 memsz=0x4ba000
(XEN) ELF: memory: 0x1000000 -> 0x2b86000
(XEN) ELF: note: GUEST_OS = "linux"
(XEN) ELF: note: GUEST_VERSION = "2.6"
(XEN) ELF: note: XEN_VERSION = "xen-3.0"
(XEN) ELF: note: VIRT_BASE = 0xffffffff80000000
(XEN) ELF: note: INIT_P2M = 0x8000000000
(XEN) ELF: note: ENTRY = 0xffffffff826cc180
(XEN) ELF: note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) ELF: note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
(XEN) ELF: note: SUPPORTED_FEATURES = 0x8801
(XEN) ELF: note: PAE_MODE = "yes"
(XEN) ELF: note: LOADER = "generic"
(XEN) ELF: note: unknown (0xd)
(XEN) ELF: note: SUSPEND_CANCEL = 0x1
(XEN) ELF: note: MOD_START_PFN = 0x1
(XEN) ELF: note: HV_START_LOW = 0xffff800000000000
(XEN) ELF: note: PADDR_OFFSET = 0
(XEN) ELF: note: PHYS32_ENTRY = 0x1000380
(XEN) ELF: Found PVH image
(XEN) ELF: addresses:
(XEN)     virt_base        = 0xffffffff80000000
(XEN)     elf_paddr_offset = 0x0
(XEN)     virt_offset      = 0xffffffff80000000
(XEN)     virt_kstart      = 0xffffffff81000000
(XEN)     virt_kend        = 0xffffffff82b86000
(XEN)     virt_entry       = 0xffffffff826cc180
(XEN)     p2m_base         = 0x8000000000
(XEN)  Xen  kernel: 64-bit, lsb, compat32
(XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2b86000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN)  Dom0 alloc.:   0000000270000000->0000000274000000 (1018277
pages to be allocated)
(XEN)  Init. ramdisk: 000000027e1a5000->00000002817ff72a
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN)  Loaded kernel: ffffffff81000000->ffffffff82b86000
(XEN)  Init. ramdisk: 0000000000000000->0000000000000000
(XEN)  Phys-Mach map: 0000008000000000->0000008000800000
(XEN)  Start info:    ffffffff82b86000->ffffffff82b864b4
(XEN)  Xenstore ring: 0000000000000000->0000000000000000
(XEN)  Console ring:  0000000000000000->0000000000000000
(XEN)  Page tables:   ffffffff82b87000->ffffffff82ba2000
(XEN)  Boot stack:    ffffffff82ba2000->ffffffff82ba3000
(XEN)  TOTAL:         ffffffff80000000->ffffffff83000000
(XEN)  ENTRY ADDRESS: ffffffff826cc180
(XEN) Dom0 has maximum 2 VCPUs
(XEN) sched_null.c:347: 1 <-- d0v1
(XEN) ELF: phdr 0 at 0xffffffff81000000 -> 0xffffffff8223c000
(XEN) ELF: phdr 1 at 0xffffffff82400000 -> 0xffffffff826a7000
(XEN) ELF: phdr 2 at 0xffffffff826a7000 -> 0xffffffff826cc000
(XEN) ELF: phdr 3 at 0xffffffff826cc000 -> 0xffffffff82911000
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
7b380000, iommu reg = ffff82c00181c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
(XEN) .........................................done.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
(XEN) Freed 2048kB init memory
(XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 0000001b from
0x00000000fee00d00 to 0x00000000fee00100
(XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 0000001b from
0x00000000fee00d00 to 0x00000000fee00900
(XEN) emul-priv-op.c:1166:d0v1 Domain attempted WRMSR 0000001b from
0x00000000fee00c00 to 0x00000000fee00000
(XEN) emul-priv-op.c:1166:d0v1 Domain attempted WRMSR 0000001b from
0x00000000fee00c00 to 0x00000000fee00800
(XEN) d0: Forcing write emulation on MFNs e0000-effff
(XEN) PCI add device 0000:00:00.0
(XEN) PCI add device 0000:00:02.0
(XEN) PCI add device 0000:00:04.0
(XEN) PCI add device 0000:00:14.0
(XEN) PCI add device 0000:00:14.2
(XEN) PCI add device 0000:00:15.0
(XEN) PCI add device 0000:00:15.1
(XEN) PCI add device 0000:00:16.0
(XEN) PCI add device 0000:00:1c.0
(XEN) PCI add device 0000:00:1c.4
(XEN) PCI add device 0000:00:1c.5
(XEN) PCI add device 0000:00:1d.0
(XEN) PCI add device 0000:00:1f.0
(XEN) PCI add device 0000:00:1f.2
(XEN) PCI add device 0000:00:1f.3
(XEN) PCI add device 0000:00:1f.4
(XEN) PCI add device 0000:3a:00.0
(XEN) PCI add device 0000:3b:00.0
(XEN) PCI add device 0000:3c:00.0
(XEN) d0: Forcing read-only access to MFN fed00
(XEN) traps.c:1570: GPF (0000): ffff82d080371462
[emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
(XEN) traps.c:1570: GPF (0000): ffff82d080371462
[emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
(XEN) traps.c:1570: GPF (0000): ffff82d080371462
[emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
(XEN) traps.c:1570: GPF (0000): ffff82d080371462
[emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
(XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
0x004280e800dd8088 to 0x004280e8009d8088
(XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
0x004280e800dd8088 to 0x004280e800dd0088
(XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
0x004280e800dd8088 to 0x004280e800dc8088
(XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
0x004280e800dd8088 to 0x004200e800dd8088
(XEN) traps.c:1570: GPF (0000): ffff82d080371462
[emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
(XEN) PCI add device 0000:01:00.0
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:02:01.0
(XEN) PCI add device 0000:02:02.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:39:00.0
(XEN) PCI remove device 0000:03:00.0
(XEN) PCI remove device 0000:02:00.0
(XEN) PCI remove device 0000:02:01.0
(XEN) PCI remove device 0000:39:00.0
(XEN) PCI remove device 0000:02:02.0
(XEN) PCI remove device 0000:01:00.0
(XEN) PCI add device 0000:01:00.0
(XEN) PCI add device 0000:02:00.0
(XEN) PCI add device 0000:02:01.0
(XEN) PCI add device 0000:02:02.0
(XEN) PCI add device 0000:03:00.0
(XEN) PCI add device 0000:39:00.0

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Jan Beulich]

>>> On 07.08.18 at 18:04, <tamas.k.lengyel@gmail.com> wrote:
>> > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
>> > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
>> > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
>> > > > 4625f3a000, iommu reg = ffff82c00181c000
>> > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>> > > > (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a

So the address here is clearly outside of any RAM:

> (XEN) EFI RAM map:
> (XEN)  0000000000000000 - 0000000000058000 (usable)
> (XEN)  0000000000058000 - 0000000000059000 (reserved)
> (XEN)  0000000000059000 - 000000000009f000 (usable)
> (XEN)  000000000009f000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000005f14d000 (usable)
> (XEN)  000000005f14d000 - 000000005f14e000 (ACPI NVS)
> (XEN)  000000005f14e000 - 000000005f14f000 (reserved)
> (XEN)  000000005f14f000 - 000000006ee89000 (usable)
> (XEN)  000000006ee89000 - 000000006f214000 (reserved)
> (XEN)  000000006f214000 - 000000006f258000 (ACPI data)
> (XEN)  000000006f258000 - 000000006f8fd000 (ACPI NVS)
> (XEN)  000000006f8fd000 - 000000006ffff000 (reserved)
> (XEN)  000000006ffff000 - 0000000070000000 (usable)
> (XEN)  0000000070000000 - 0000000078000000 (reserved)
> (XEN)  0000000078000000 - 0000000078600000 (usable)
> (XEN)  0000000078600000 - 000000007c800000 (reserved)
> (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> (XEN)  00000000fe000000 - 00000000fe011000 (reserved)
> (XEN)  00000000fec00000 - 00000000fec01000 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 0000000281800000 (usable)

And it's also outside of any of the GPU device's BARs. Makes
me wonder what the graphics driver is doing.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
>
> On Tue, Aug 7, 2018 at 9:09 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> >
> > On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote:
> > > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> > > >
> > > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
> > > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> > > > > >
> > > > > > Hello,
> > > > > >
> > > > > > The following series implement a workaround for missing RMRR
> > > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> > > > > > option.
> > > > > >
> > > > > > The PVH workaround identity maps all regions marked as reserved in the
> > > > > > memory map.
> > > > > >
> > > > > > Note that this workaround is enabled by default on Intel hardware. It's
> > > > > > also available to AMD hardware, although it's disabled by default in
> > > > > > that case.
> > > > > >
> > > > > > The series can be found at:
> > > > > >
> > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
> > > > > >
> > > > > > Thanks, Roger.
> > > > > > Roger Pau Monne (4):
> > > > > >   iommu: introduce dom0-iommu option
> > > > > >   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
> > > > > >   dom0/pvh: change the order of the MMCFG initialization
> > > > > >   x86/iommu: add reserved dom0-iommu option to map reserved memory
> > > > > >     ranges
> > > > > >
> > > > > >  docs/misc/xen-command-line.markdown         | 47 +++++++++++
> > > > > >  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
> > > > > >  xen/arch/x86/hvm/io.c                       |  5 ++
> > > > > >  xen/arch/x86/x86_64/mm.c                    |  3 +-
> > > > > >  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
> > > > > >  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
> > > > > >  xen/drivers/passthrough/arm/iommu.c         |  4 +
> > > > > >  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
> > > > > >  xen/drivers/passthrough/vtd/extern.h        |  2 -
> > > > > >  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
> > > > > >  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
> > > > > >  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
> > > > > >  xen/include/asm-x86/hvm/io.h                |  3 +
> > > > > >  xen/include/xen/iommu.h                     |  8 +-
> > > > > >  14 files changed, 240 insertions(+), 86 deletions(-)
> > > > > >
> > > > > > --
> > > > >
> > > > > Hi Roger,
> > > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
> > > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
> > > > > keyboard of the laptop stopped working, it works no problem with just
> > > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
> > > > > running for a minute or two the system starts to slow down to the
> > > > > point where it becomes unresponsive. The xl dmesg log is filled with
> > > > > this error:
> > > > >
> > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> > > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> > > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> > > > > 4625f3a000, iommu reg = ffff82c00181c000
> > > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> > > > > (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
> > > >
> > > > Is the gmfn always the same (0x4625f3a)?
> > > >
> > > > > (XEN)     root_entry[00] = 273a18001
> > > > > (XEN)     context[10] = 2_27ba35001
> > > > > (XEN)     l4[000] = 9c0000027ba34107
> > > > > (XEN)     l3[118] = 8000000000000000
> > > > > (XEN)     l3[118] not present
> > > >
> > > > Can you also paste the full xl dmesg log? I'm specially interested in
> > > > the memory map of the machine which is printed quite early during Xen
> > > > boot.
> > > >
> > >
> > > Unfortunately I don't have serial access on this laptop and "xl dmesg"
> > > gets completely filled with that error so the beginning of the log is
> > > lost by the time I get a terminal in dom0.
> >
> > You can get the memory map while booting in PV mode, it's going to be
> > exactly the same regardless of whether Dom0 is PV or PVH.
>
> This is the PV dmesg:
>
> (XEN) Xen version 4.12-unstable (dr@) (gcc (Debian 7.3.0-19) 7.3.0)
> debug=y  Mon Aug  6 13:42:42 MDT 2018
> (XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f
> (XEN) Bootloader: EFI
> (XEN) Command line: loglvl=all guest_loglvl=all
> dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null console=vga
> (XEN) Xen image load base address: 0x5a200000
> (XEN) Video information:
> (XEN)  VGA is graphics mode 3200x1800, 32 bpp
> (XEN) Disc information:
> (XEN)  Found 0 MBR signatures
> (XEN)  Found 1 EDD information structures
> (XEN) EFI RAM map:
> (XEN)  0000000000000000 - 0000000000058000 (usable)
> (XEN)  0000000000058000 - 0000000000059000 (reserved)
> (XEN)  0000000000059000 - 000000000009f000 (usable)
> (XEN)  000000000009f000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000005f14d000 (usable)
> (XEN)  000000005f14d000 - 000000005f14e000 (ACPI NVS)
> (XEN)  000000005f14e000 - 000000005f14f000 (reserved)
> (XEN)  000000005f14f000 - 000000006ee89000 (usable)
> (XEN)  000000006ee89000 - 000000006f214000 (reserved)
> (XEN)  000000006f214000 - 000000006f258000 (ACPI data)
> (XEN)  000000006f258000 - 000000006f8fd000 (ACPI NVS)
> (XEN)  000000006f8fd000 - 000000006ffff000 (reserved)
> (XEN)  000000006ffff000 - 0000000070000000 (usable)
> (XEN)  0000000070000000 - 0000000078000000 (reserved)
> (XEN)  0000000078000000 - 0000000078600000 (usable)
> (XEN)  0000000078600000 - 000000007c800000 (reserved)
> (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> (XEN)  00000000fe000000 - 00000000fe011000 (reserved)
> (XEN)  00000000fec00000 - 00000000fec01000 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 0000000281800000 (usable)
> (XEN) ACPI: RSDP 6F221000, 0024 (r2 DELL  )
> (XEN) ACPI: XSDT 6F2210C8, 0114 (r1 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: FACP 6F246918, 010C (r5 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: DSDT 6F221270, 256A6 (r2 DELL    CBX3     1072009 INTL 20160422)
> (XEN) ACPI: FACS 6F8FA180, 0040
> (XEN) ACPI: APIC 6F246A28, 0084 (r3 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: FPDT 6F246AB0, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: FIDT 6F246AF8, 00AC (r1 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: MCFG 6F246BA8, 003C (r1 DELL    CBX3     1072009 MSFT       97)
> (XEN) ACPI: HPET 6F246BE8, 0038 (r1 DELL    CBX3     1072009 AMI.    5000B)
> (XEN) ACPI: SSDT 6F246C20, 0359 (r1 SataRe SataTabl     1000 INTL 20160422)
> (XEN) ACPI: BOOT 6F246F80, 0028 (r1 DELL   CBX3      1072009 AMI     10013)
> (XEN) ACPI: SSDT 6F246FA8, 12CF (r2 SaSsdt  SaSsdt      3000 INTL 20160422)
> (XEN) ACPI: HPET 6F248278, 0038 (r1 INTEL   KBL-ULT        1 MSFT       5F)
> (XEN) ACPI: SSDT 6F2482B0, 0D84 (r2  INTEL xh_rvp07        0 INTL 20160422)
> (XEN) ACPI: UEFI 6F249038, 0042 (r1                        0             0)
> (XEN) ACPI: SSDT 6F249080, 0EDE (r2 CpuRef  CpuSsdt     3000 INTL 20160422)
> (XEN) ACPI: LPIT 6F249F60, 0094 (r1 INTEL   KBL-ULT        0 MSFT       5F)
> (XEN) ACPI: WSMT 6F249FF8, 0028 (r1 DELL    CBX3           0 MSFT       5F)
> (XEN) ACPI: SSDT 6F24A020, 0161 (r2 INTEL    HdaDsp        0 INTL 20160422)
> (XEN) ACPI: SSDT 6F24A188, 029F (r2 INTEL  sensrhub        0 INTL 20160422)
> (XEN) ACPI: SSDT 6F24A428, 3002 (r2 INTEL  PtidDevc     1000 INTL 20160422)
> (XEN) ACPI: SSDT 6F24D430, 00DB (r2 INTEL  TbtTypeC        0 INTL 20160422)
> (XEN) ACPI: DBGP 6F24D510, 0034 (r1 INTEL                  2 MSFT       5F)
> (XEN) ACPI: DBG2 6F24D548, 0054 (r0 INTEL                  2 MSFT       5F)
> (XEN) ACPI: SSDT 6F24D5A0, 07AC (r2 INTEL  UsbCTabl     1000 INTL 20160422)
> (XEN) ACPI: SSDT 6F24DD50, 84F1 (r2 DptfTa DptfTabl     1000 INTL 20160422)
> (XEN) ACPI: MSDM 6F256248, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
> (XEN) ACPI: SLIC 6F2562A0, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
> (XEN) ACPI: NHLT 6F256418, 002D (r0 INTEL  EDK2            2       1000013)
> (XEN) ACPI: BGRT 6F256448, 0038 (r0     ��           1072009 AMI     10013)
> (XEN) ACPI: TPM2 6F256480, 0034 (r3        Tpm2Tabl        1 AMI         0)
> (XEN) ACPI: ASF! 6F2564B8, 00A0 (r32 INTEL       HCG        1 TFSM    F4240)
> (XEN) ACPI: DMAR 6F256558, 00F0 (r1 INTEL      KBL         1 INTL        1)
> (XEN) System RAM: 7948MB (8138904kB)
> (XEN) No NUMA configuration found
> (XEN) Faking a node at 0000000000000000-0000000281800000
> (XEN) Domain heap initialised
> (XEN) vesafb: framebuffer at 0x90000000, mapped to 0xffff82c000201000,
> using 22528k, total 22528k
> (XEN) vesafb: mode is 3200x1800x32, linelength=12800, font 8x16
> (XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
> (XEN) Couldn't initialize a 3200x1800 framebuffer early.
> (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 142 (0x8e), Stepping 9
> (raw 000806e9)
> (XEN) SMBIOS 3.0 present.
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1808 (32 bits)
> (XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
> (XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> 6f8fa180/0000000000000000, using 32
> (XEN) ACPI:             wakeup_vec[6f8fa18c], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
> (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) ACPI: IRQ9 used by override.
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
> (XEN) [VT-D]dmar.c:851: Ignore unknown DMAR structure type (0x4)
> (XEN) [VT-D]dmar.c:851: Ignore unknown DMAR structure type (0x4)
> (XEN) ERST table was not found
> (XEN) ACPI: BGRT: invalidating v1 image at 0x6ad04018
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
> (XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
> (XEN) Switched to APIC driver x2apic_cluster.
> (XEN) xstate: size: 0x440 and states: 0x1f
> (XEN) mce_intel.c:780: MCA Capability: firstbank 0, extended MCE MSR
> 0, BCAST, CMCI
> (XEN) CPU0: Intel machine check reporting enabled
> (XEN) Speculative mitigation facilities:
> (XEN)   Hardware features: IBRS/IBPB STIBP
> (XEN)   Compiled-in support: INDIRECT_THUNK
> (XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+, Other: IBPB
> (XEN)   Support for VMs: PV: MSR_SPEC_CTRL RSB EAGER_FPU, HVM:
> MSR_SPEC_CTRL RSB EAGER_FPU
> (XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled
> (XEN) Using scheduler: null Scheduler (null)
> (XEN) Initializing null scheduler
> (XEN) WARNING: This is experimental software in development.
> (XEN) Use at your own risk.
> (XEN) Platform timer is 23.999MHz HPET
> (XEN) Detected 2904.041 MHz processor.
> (XEN) EFI memory map:
> (XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
> (XEN)  0000000008000-0000000057fff type=7 attr=000000000000000f
> (XEN)  0000000058000-0000000058fff type=0 attr=000000000000000f
> (XEN)  0000000059000-000000005cfff type=7 attr=000000000000000f
> (XEN)  000000005d000-000000005efff type=2 attr=000000000000000f
> (XEN)  000000005f000-000000005ffff type=4 attr=000000000000000f
> (XEN)  0000000060000-000000009efff type=3 attr=000000000000000f
> (XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
> (XEN)  0000000100000-000003f225fff type=7 attr=000000000000000f
> (XEN)  000003f226000-000003f265fff type=4 attr=000000000000000f
> (XEN)  000003f266000-00000561bbfff type=7 attr=000000000000000f
> (XEN)  00000561bc000-000005a03afff type=2 attr=000000000000000f
> (XEN)  000005a03b000-000005b23afff type=1 attr=000000000000000f
> (XEN)  000005b23b000-000005b3e5fff type=2 attr=000000000000000f
> (XEN)  000005b3e6000-000005b3e9fff type=7 attr=000000000000000f
> (XEN)  000005b3ea000-000005b415fff type=2 attr=000000000000000f
> (XEN)  000005b416000-000005b420fff type=7 attr=000000000000000f
> (XEN)  000005b421000-000005b458fff type=2 attr=000000000000000f
> (XEN)  000005b459000-000005b45afff type=7 attr=000000000000000f
> (XEN)  000005b45b000-000005b462fff type=2 attr=000000000000000f
> (XEN)  000005b463000-000005b464fff type=7 attr=000000000000000f
> (XEN)  000005b465000-000005d2e0fff type=2 attr=000000000000000f
> (XEN)  000005d2e1000-000005d325fff type=1 attr=000000000000000f
> (XEN)  000005d326000-000005f14cfff type=4 attr=000000000000000f
> (XEN)  000005f14d000-000005f14dfff type=10 attr=000000000000000f
> (XEN)  000005f14e000-000005f14efff type=6 attr=800000000000000f
> (XEN)  000005f14f000-000005f215fff type=4 attr=000000000000000f
> (XEN)  000005f216000-000005f224fff type=2 attr=000000000000000f
> (XEN)  000005f225000-000006dfe6fff type=4 attr=000000000000000f
> (XEN)  000006dfe7000-000006e2fafff type=7 attr=000000000000000f
> (XEN)  000006e2fb000-000006ee88fff type=3 attr=000000000000000f
> (XEN)  000006ee89000-000006f213fff type=0 attr=000000000000000f
> (XEN)  000006f214000-000006f257fff type=9 attr=000000000000000f
> (XEN)  000006f258000-000006f8fcfff type=10 attr=000000000000000f
> (XEN)  000006f8fd000-000006ff1ffff type=6 attr=800000000000000f
> (XEN)  000006ff20000-000006fffefff type=5 attr=800000000000000f
> (XEN)  000006ffff000-000006fffffff type=4 attr=000000000000000f
> (XEN)  0000078000000-00000785fffff type=7 attr=000000000000000f
> (XEN)  0000100000000-00002817fffff type=7 attr=000000000000000f
> (XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
> (XEN)  0000070000000-0000077ffffff type=0 attr=070000000000000f
> (XEN)  0000078600000-000007c7fffff type=0 attr=0000000000000000
> (XEN)  00000e0000000-00000efffffff type=11 attr=8000000000000000
> (XEN) Unknown cachability for MFNs 0xe0000-0xeffff
> (XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
> (XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
> (XEN)  00000fee00000-00000fee00fff type=11 attr=800000000000100d
> (XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d0806739b0 -> ffff82d0806757c4
> (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
> (XEN) PCI: MCFG area at e0000000 reserved in E820
> (XEN) PCI: Using MCFG for segment 0000 bus 00-ff
> (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) Intel VT-d Snoop Control not enabled.
> (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> (XEN) Intel VT-d Queued Invalidation enabled.
> (XEN) Intel VT-d Interrupt Remapping enabled.
> (XEN) Intel VT-d Posted Interrupt not enabled.
> (XEN) Intel VT-d Shared EPT tables enabled.
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) Interrupt remapping enabled
> (XEN) nr_sockets: 1
> (XEN) Enabled directed EOI with ioapic_ack_old on!
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using old ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=0 pin2=0
> (XEN) TSC deadline timer enabled
> (XEN) Allocated console ring of 32 KiB.
> (XEN) mwait-idle: MWAIT substates: 0x11142120
> (XEN) mwait-idle: v0.4.1 model 0x8e
> (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) VMX: Supported advanced features:
> (XEN)  - APIC MMIO access virtualisation
> (XEN)  - APIC TPR shadow
> (XEN)  - Extended Page Tables (EPT)
> (XEN)  - Virtual-Processor Identifiers (VPID)
> (XEN)  - Virtual NMI
> (XEN)  - MSR direct-access bitmap
> (XEN)  - Unrestricted Guest
> (XEN)  - VM Functions
> (XEN)  - Virtualisation Exceptions
> (XEN)  - Page Modification Logging
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) Brought up 4 CPUs
> (XEN) build-id: 1e3eb01a83018530b8a66809744145e8
> (XEN) Running stub recovery selftests...
> (XEN) traps.c:1570: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] ->
> ffff82d08037f3f2
> (XEN) traps.c:755: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] ->
> ffff82d08037f3f2
> (XEN) traps.c:1097: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] ->
> ffff82d08037f3f2
> (XEN) ACPI sleep modes: S3
> (XEN) VPMU: disabled
> (XEN) mcheck_poll: Machine check polling timer started.
> (XEN) Dom0 has maximum 504 PIRQs
> (XEN) sched_null.c:347: 0 <-- d0v0
> (XEN) NX (Execute Disable) protection active
> (XEN) *** Building a PV Dom0 ***
> (XEN) ELF: phdr: paddr=0x1000000 memsz=0x123c000
> (XEN) ELF: phdr: paddr=0x2400000 memsz=0x2a7000
> (XEN) ELF: phdr: paddr=0x26a7000 memsz=0x25000
> (XEN) ELF: phdr: paddr=0x26cc000 memsz=0x4ba000
> (XEN) ELF: memory: 0x1000000 -> 0x2b86000
> (XEN) ELF: note: GUEST_OS = "linux"
> (XEN) ELF: note: GUEST_VERSION = "2.6"
> (XEN) ELF: note: XEN_VERSION = "xen-3.0"
> (XEN) ELF: note: VIRT_BASE = 0xffffffff80000000
> (XEN) ELF: note: INIT_P2M = 0x8000000000
> (XEN) ELF: note: ENTRY = 0xffffffff826cc180
> (XEN) ELF: note: HYPERCALL_PAGE = 0xffffffff81001000
> (XEN) ELF: note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
> (XEN) ELF: note: SUPPORTED_FEATURES = 0x8801
> (XEN) ELF: note: PAE_MODE = "yes"
> (XEN) ELF: note: LOADER = "generic"
> (XEN) ELF: note: unknown (0xd)
> (XEN) ELF: note: SUSPEND_CANCEL = 0x1
> (XEN) ELF: note: MOD_START_PFN = 0x1
> (XEN) ELF: note: HV_START_LOW = 0xffff800000000000
> (XEN) ELF: note: PADDR_OFFSET = 0
> (XEN) ELF: note: PHYS32_ENTRY = 0x1000380
> (XEN) ELF: Found PVH image
> (XEN) ELF: addresses:
> (XEN)     virt_base        = 0xffffffff80000000
> (XEN)     elf_paddr_offset = 0x0
> (XEN)     virt_offset      = 0xffffffff80000000
> (XEN)     virt_kstart      = 0xffffffff81000000
> (XEN)     virt_kend        = 0xffffffff82b86000
> (XEN)     virt_entry       = 0xffffffff826cc180
> (XEN)     p2m_base         = 0x8000000000
> (XEN)  Xen  kernel: 64-bit, lsb, compat32
> (XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2b86000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN)  Dom0 alloc.:   0000000270000000->0000000274000000 (1018277
> pages to be allocated)
> (XEN)  Init. ramdisk: 000000027e1a5000->00000002817ff72a
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN)  Loaded kernel: ffffffff81000000->ffffffff82b86000
> (XEN)  Init. ramdisk: 0000000000000000->0000000000000000
> (XEN)  Phys-Mach map: 0000008000000000->0000008000800000
> (XEN)  Start info:    ffffffff82b86000->ffffffff82b864b4
> (XEN)  Xenstore ring: 0000000000000000->0000000000000000
> (XEN)  Console ring:  0000000000000000->0000000000000000
> (XEN)  Page tables:   ffffffff82b87000->ffffffff82ba2000
> (XEN)  Boot stack:    ffffffff82ba2000->ffffffff82ba3000
> (XEN)  TOTAL:         ffffffff80000000->ffffffff83000000
> (XEN)  ENTRY ADDRESS: ffffffff826cc180
> (XEN) Dom0 has maximum 2 VCPUs
> (XEN) sched_null.c:347: 1 <-- d0v1
> (XEN) ELF: phdr 0 at 0xffffffff81000000 -> 0xffffffff8223c000
> (XEN) ELF: phdr 1 at 0xffffffff82400000 -> 0xffffffff826a7000
> (XEN) ELF: phdr 2 at 0xffffffff826a7000 -> 0xffffffff826cc000
> (XEN) ELF: phdr 3 at 0xffffffff826cc000 -> 0xffffffff82911000
> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> 7b380000, iommu reg = ffff82c00181c000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
> (XEN) .........................................done.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
> input to Xen)
> (XEN) Freed 2048kB init memory
> (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 0000001b from
> 0x00000000fee00d00 to 0x00000000fee00100
> (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 0000001b from
> 0x00000000fee00d00 to 0x00000000fee00900
> (XEN) emul-priv-op.c:1166:d0v1 Domain attempted WRMSR 0000001b from
> 0x00000000fee00c00 to 0x00000000fee00000
> (XEN) emul-priv-op.c:1166:d0v1 Domain attempted WRMSR 0000001b from
> 0x00000000fee00c00 to 0x00000000fee00800
> (XEN) d0: Forcing write emulation on MFNs e0000-effff
> (XEN) PCI add device 0000:00:00.0
> (XEN) PCI add device 0000:00:02.0
> (XEN) PCI add device 0000:00:04.0
> (XEN) PCI add device 0000:00:14.0
> (XEN) PCI add device 0000:00:14.2
> (XEN) PCI add device 0000:00:15.0
> (XEN) PCI add device 0000:00:15.1
> (XEN) PCI add device 0000:00:16.0
> (XEN) PCI add device 0000:00:1c.0
> (XEN) PCI add device 0000:00:1c.4
> (XEN) PCI add device 0000:00:1c.5
> (XEN) PCI add device 0000:00:1d.0
> (XEN) PCI add device 0000:00:1f.0
> (XEN) PCI add device 0000:00:1f.2
> (XEN) PCI add device 0000:00:1f.3
> (XEN) PCI add device 0000:00:1f.4
> (XEN) PCI add device 0000:3a:00.0
> (XEN) PCI add device 0000:3b:00.0
> (XEN) PCI add device 0000:3c:00.0
> (XEN) d0: Forcing read-only access to MFN fed00
> (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> 0x004280e800dd8088 to 0x004280e8009d8088
> (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> 0x004280e800dd8088 to 0x004280e800dd0088
> (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> 0x004280e800dd8088 to 0x004280e800dc8088
> (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> 0x004280e800dd8088 to 0x004200e800dd8088
> (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> (XEN) PCI add device 0000:01:00.0
> (XEN) PCI add device 0000:02:00.0
> (XEN) PCI add device 0000:02:01.0
> (XEN) PCI add device 0000:02:02.0
> (XEN) PCI add device 0000:03:00.0
> (XEN) PCI add device 0000:39:00.0
> (XEN) PCI remove device 0000:03:00.0
> (XEN) PCI remove device 0000:02:00.0
> (XEN) PCI remove device 0000:02:01.0
> (XEN) PCI remove device 0000:39:00.0
> (XEN) PCI remove device 0000:02:02.0
> (XEN) PCI remove device 0000:01:00.0
> (XEN) PCI add device 0000:01:00.0
> (XEN) PCI add device 0000:02:00.0
> (XEN) PCI add device 0000:02:01.0
> (XEN) PCI add device 0000:02:02.0
> (XEN) PCI add device 0000:03:00.0
> (XEN) PCI add device 0000:39:00.0


This is the log on another Dell laptop with which I have serial access
and it has the same error, physical keyboard/touchpad doesn't work but
attaching one via USB does:

(XEN) Xen version 4.12-unstable (drt@) (gcc (Ubuntu 7.3.0-16ubuntu3)
7.3.0) debug=y  Mon Aug  6 16:40:02 MDT 2018
(XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f
(XEN) Bootloader: EFI
(XEN) Command line: loglvl=all guest_loglvl=all
dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null dom0=pvh
iommu=required,debug dom0-iommu=relaxed console=vga,com1 com1=1152
00,8n1,pci
(XEN) Xen image load base address: 0x99200000
(XEN) Video information:
(XEN)  VGA is graphics mode 1920x1080, 32 bpp
(XEN) Disc information:
(XEN)  Found 0 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) EFI RAM map:
(XEN)  0000000000000000 - 0000000000058000 (usable)
(XEN)  0000000000058000 - 0000000000059000 (reserved)
(XEN)  0000000000059000 - 000000000009f000 (usable)
(XEN)  000000000009f000 - 0000000000100000 (reserved)
(XEN)  0000000000100000 - 000000009cba2000 (usable)
(XEN)  000000009cba2000 - 000000009cba3000 (ACPI NVS)
(XEN)  000000009cba3000 - 000000009cbcd000 (reserved)
(XEN)  000000009cbcd000 - 00000000ab080000 (usable)
(XEN)  00000000ab080000 - 00000000ab3f5000 (reserved)
(XEN)  00000000ab3f5000 - 00000000ab42e000 (ACPI data)
(XEN)  00000000ab42e000 - 00000000aba6c000 (ACPI NVS)
(XEN)  00000000aba6c000 - 00000000ac0ff000 (reserved)
(XEN)  00000000ac0ff000 - 00000000ac100000 (usable)
(XEN)  00000000ac100000 - 00000000b0000000 (reserved)
(XEN)  00000000f0000000 - 00000000f8000000 (reserved)
(XEN)  00000000fe000000 - 00000000fe011000 (reserved)
(XEN)  00000000fec00000 - 00000000fec01000 (reserved)
(XEN)  00000000fed20000 - 00000000fed80000 (reserved)
(XEN)  00000000fee00000 - 00000000fee01000 (reserved)
(XEN)  00000000ff000000 - 0000000100000000 (reserved)
(XEN)  0000000100000000 - 000000044e000000 (usable)
(XEN) ACPI: RSDP AB400000, 0024 (r2 DELL  )
(XEN) ACPI: XSDT AB4000B0, 00E4 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FACP AB41FB78, 010C (r5 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: DSDT AB400228, 1F94F (r2 DELL    CBX3     1072009 INTL 20120913)
(XEN) ACPI: FACS ABA69E80, 0040
(XEN) ACPI: UEFI AB420000, 063A (r1  INTEL RstSataE        0   ��        0)
(XEN) ACPI: APIC AB420640, 0084 (r3 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: FPDT AB4206C8, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
(XEN) ACPI: MCFG AB420710, 003C (r1 DELL    CBX3     1072009 MSFT       97)
(XEN) ACPI: HPET AB420750, 0038 (r1 DELL    CBX3     1072009 AMI.    5000B)
(XEN) ACPI: SSDT AB420788, 046D (r1 SataRe SataTabl     1000 INTL 20120913)
(XEN) ACPI: LPIT AB420BF8, 0094 (r1 INTEL   SKL-ULT        0 MSFT       5F)
(XEN) ACPI: SSDT AB420C90, 0248 (r2 INTEL  sensrhub        0 INTL 20120913)
(XEN) ACPI: SSDT AB420ED8, 2BAE (r2 INTEL  PtidDevc     1000 INTL 20120913)
(XEN) ACPI: SSDT AB423A88, 0814 (r2 INTEL  xh_rvp07        0 INTL 20120913)
(XEN) ACPI: DBGP AB4242A0, 0034 (r1 INTEL                  0 MSFT       5F)
(XEN) ACPI: DBG2 AB4242D8, 0054 (r0 INTEL                  0 MSFT       5F)
(XEN) ACPI: SSDT AB424330, 35DA (r2 SaSsdt  SaSsdt      3000 INTL 20120913)
(XEN) ACPI: UEFI AB427910, 0042 (r1                        0             0)
(XEN) ACPI: SSDT AB427958, 0E73 (r2 CpuRef  CpuSsdt     3000 INTL 20120913)
(XEN) ACPI: SSDT AB4287D0, 3BFD (r2 DptfTa DptfTabl     1000 INTL 20120913)
(XEN) ACPI: MSDM AB42C3D0, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
(XEN) ACPI: SLIC AB42C428, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
(XEN) ACPI: TCPA AB42C5A0, 0032 (r2 ALASKA  NAPAASF        0 MSFT  1000013)
(XEN) ACPI: DMAR AB42C5D8, 00A8 (r1 INTEL      SKL         1 INTL        1)
(XEN) ACPI: UEFI AB42C680, 005C (r1  INTEL RstSataV        0   ��        0)
(XEN) ACPI: ASF! AB42C6E0, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
(XEN) ACPI: BGRT AB42C788, 0038 (r0     ��           1072009 AMI     10013)
(XEN) System RAM: 16271MB (16662480kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-000000044e000000
(XEN) Domain heap initialised
(XEN) vesafb: framebuffer at 0xd0000000, mapped to 0xffff82c000201000,
using 8128k, total 8128k
(XEN) vesafb: mode is 1920x1080x32, linelength=7680, font 8x16
(XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
(XEN) Couldn't initialize a 1920x1080 framebuffer early.
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 78 (0x4e), Stepping 3
(raw 000406e3)
(XEN) SMBIOS 3.0 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1808 (32 bits)
(XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
(XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT -
aba69e80/0000000000000000, using 32
(XEN) ACPI:             wakeup_vec[aba69e8c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]Host address width 39
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D]  dmaru->address = fed90000
(XEN) [VT-D]drhd->address = fed90000 iommu->reg = ffff82c000a0c000
(XEN) [VT-D]cap = 1c0000c40660462 ecap = 7e3ff0505e
(XEN) [VT-D] endpoint: 0000:00:02.0
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D]  dmaru->address = fed91000
(XEN) [VT-D]drhd->address = fed91000 iommu->reg = ffff82c000a0e000
(XEN) [VT-D]cap = d2008c40660462 ecap = f050da
(XEN) [VT-D] IOAPIC: 0000:f0:1f.0
(XEN) [VT-D] MSI HPET: 0000:00:1f.0
(XEN) [VT-D]  flags: INCLUDE_ALL
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:14.0
(XEN) [VT-D]dmar.c:638:   RMRR region: base_addr ab198000 end_addr ab1b7fff
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:02.0
(XEN) [VT-D]dmar.c:638:   RMRR region: base_addr ad800000 end_addr afffffff
(XEN) ERST table was not found
(XEN) ACPI: BGRT: invalidating v1 image at 0xa837d018
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
(XEN) Not enabling x2APIC (upon firmware request)
(XEN) xstate: size: 0x440 and states: 0x1f
(XEN) mce_intel.c:780: MCA Capability: firstbank 0, extended MCE MSR
0, BCAST, CMCI
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Speculative mitigation facilities:
(XEN)   Hardware features: IBRS/IBPB STIBP
(XEN)   Compiled-in support: INDIRECT_THUNK
(XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+, Other: IBPB
(XEN)   Support for VMs: PV: MSR_SPEC_CTRL RSB EAGER_FPU, HVM:
MSR_SPEC_CTRL RSB EAGER_FPU
(XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled
(XEN) Using scheduler: null Scheduler (null)
(XEN) Initializing null scheduler
(XEN) WARNING: This is experimental software in development.
(XEN) Use at your own risk.
(XEN) Platform timer is 23.999MHz HPET
(XEN) Detected 2808.069 MHz processor.
(XEN) EFI memory map:
(XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
(XEN)  0000000008000-0000000057fff type=7 attr=000000000000000f
(XEN)  0000000058000-0000000058fff type=0 attr=000000000000000f
(XEN)  0000000059000-000000005cfff type=7 attr=000000000000000f
(XEN)  000000005d000-000000005efff type=2 attr=000000000000000f
(XEN)  000000005f000-000000005ffff type=4 attr=000000000000000f
(XEN)  0000000060000-000000009efff type=3 attr=000000000000000f
(XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
(XEN)  0000000100000-000007cc3cfff type=7 attr=000000000000000f
(XEN)  000007cc3d000-000007cc7cfff type=4 attr=000000000000000f
(XEN)  000007cc7d000-0000095330fff type=7 attr=000000000000000f
(XEN)  0000095331000-00000990c7fff type=2 attr=000000000000000f
(XEN)  00000990c8000-000009a3b0fff type=1 attr=000000000000000f
(XEN)  000009a3b1000-000009a407fff type=2 attr=000000000000000f
(XEN)  000009a408000-000009a408fff type=7 attr=000000000000000f
(XEN)  000009a409000-000009a431fff type=2 attr=000000000000000f
(XEN)  000009a432000-000009a432fff type=7 attr=000000000000000f
(XEN)  000009a433000-000009a440fff type=2 attr=000000000000000f
(XEN)  000009a441000-000009a442fff type=7 attr=000000000000000f
(XEN)  000009a443000-000009a44dfff type=2 attr=000000000000000f
(XEN)  000009a44e000-000009a44ffff type=7 attr=000000000000000f
(XEN)  000009a450000-000009b062fff type=2 attr=000000000000000f
(XEN)  000009b063000-000009b0a7fff type=1 attr=000000000000000f
(XEN)  000009b0a8000-000009cba1fff type=4 attr=000000000000000f
(XEN)  000009cba2000-000009cba2fff type=10 attr=000000000000000f
(XEN)  000009cba3000-000009cbccfff type=6 attr=800000000000000f
(XEN)  000009cbcd000-000009cc2cfff type=4 attr=000000000000000f
(XEN)  000009cc2d000-000009cc3bfff type=2 attr=000000000000000f
(XEN)  000009cc3c000-00000aa767fff type=4 attr=000000000000000f
(XEN)  00000aa768000-00000aa97ffff type=7 attr=000000000000000f
(XEN)  00000aa980000-00000ab07ffff type=3 attr=000000000000000f
(XEN)  00000ab080000-00000ab3f4fff type=0 attr=000000000000000f
(XEN)  00000ab3f5000-00000ab42dfff type=9 attr=000000000000000f
(XEN)  00000ab42e000-00000aba6bfff type=10 attr=000000000000000f
(XEN)  00000aba6c000-00000ac026fff type=6 attr=800000000000000f
(XEN)  00000ac027000-00000ac0fefff type=5 attr=800000000000000f
(XEN)  00000ac0ff000-00000ac0fffff type=4 attr=000000000000000f
(XEN)  0000100000000-000044dffffff type=7 attr=000000000000000f
(XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
(XEN)  00000ac100000-00000afffffff type=0 attr=0000000000000000
(XEN)  00000f0000000-00000f7ffffff type=11 attr=800000000000100d
(XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
(XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
(XEN)  00000fed20000-00000fed7ffff type=0 attr=0000000000000000
(XEN)  00000fee00000-00000fee00fff type=11 attr=800000000000100d
(XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d
(XEN) Initing memory sharing.
(XEN) alt table ffff82d080672850 -> ffff82d080674656
(XEN) spurious 8259A interrupt: IRQ7.
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control not enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN)  - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 1
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN)  -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=0 pin2=0
(XEN) TSC deadline timer enabled
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x11142120
(XEN) mwait-idle: v0.4.1 model 0x4e
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) VMX: Supported advanced features:
(XEN)  - APIC MMIO access virtualisation
(XEN)  - APIC TPR shadow
(XEN)  - Extended Page Tables (EPT)
(XEN)  - Virtual-Processor Identifiers (VPID)
(XEN)  - Virtual NMI
(XEN)  - MSR direct-access bitmap
(XEN)  - Unrestricted Guest
(XEN)  - VMCS shadowing
(XEN)  - VM Functions
(XEN)  - Virtualisation Exceptions
(XEN)  - Page Modification Logging
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) mwait-idle: max C-state count of 8 reached
(XEN) Brought up 4 CPUs
(XEN) build-id: 8b75b598bb19e50dd6b62cdd229ab5d3
(XEN) Running stub recovery selftests...
(XEN) traps.c:1570: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] ->
ffff82d08037d3f2
(XEN) traps.c:755: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] ->
ffff82d08037d3f2
(XEN) traps.c:1097: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] ->
ffff82d08037d3f2
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 504 PIRQs
(XEN) sched_null.c:347: 0 <-- d0v0
(XEN) NX (Execute Disable) protection active
(XEN) *** Building a PVH Dom0 ***
(XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map
(XEN) Bogus DMIBAR 0xfed18001 on 0000:00:00.0
(XEN) [VT-D]d0:PCIe: map 0000:00:02.0
(XEN) [VT-D]d0:PCI: map 0000:00:04.0
(XEN) [VT-D]d0:PCI: map 0000:00:14.0
(XEN) [VT-D]d0:PCI: map 0000:00:14.2
(XEN) [VT-D]d0:PCI: map 0000:00:16.0
(XEN) [VT-D]d0:PCI: map 0000:00:16.3
(XEN) [VT-D]d0:PCI: map 0000:00:17.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.2
(XEN) [VT-D]d0:PCI: map 0000:00:1f.3
(XEN) [VT-D]d0:PCI: map 0000:00:1f.4
(XEN) [VT-D]d0:PCI: map 0000:00:1f.6
(XEN) [VT-D]d0:PCIe: map 0000:01:00.0
(XEN) [VT-D]d0:PCIe: map 0000:02:00.0
(XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c000a0c000
(XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c000a0e000
(XEN) sched_null.c:347: 1 <-- d0v1
(XEN) WARNING: PVH is an experimental mode with limited functionality
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
(XEN) .....................................................................done.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
(XEN) Freed 2048kB init memory
(XEN) Dom0 callback via changed to Direct Vector 0xf3
[    0.000000] Xen Platform PCI: unrecognised magic value
(XEN) d0: bind: m_gsi=9 g_gsi=9
[    0.127065] pci 0000:00:00.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.136467] pci 0000:00:02.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.137007] pci 0000:00:04.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.139389] pci 0000:00:14.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.140103] pci 0000:00:14.2: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.140632] pci 0000:00:16.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.141133] pci 0000:00:16.3: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.142337] pci 0000:00:17.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.143120] pci 0000:00:1c.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.143866] pci 0000:00:1d.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.144407] pci 0000:00:1f.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.145467] pci 0000:00:1f.2: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.147911] pci 0000:00:1f.3: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.148648] pci 0000:00:1f.4: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.152896] pci 0000:00:1f.6: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.165264] pci 0000:01:00.0: Failed to add - passthrough or
MSI/MSI-X might fail!
[    0.169878] pci 0000:02:00.0: Failed to add - passthrough or
MSI/MSI-X might fail!
....
(XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
(XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
428f926000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
(XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
(XEN)     root_entry[00] = 43aaae001
(XEN)     context[10] = 2_43cf92001
(XEN)     l4[000] = 9c0000043cf91107
(XEN)     l3[10a] = 8000000000000000
(XEN)     l3[10a] not present

The fault is repeated a million times per second and the system is
pretty much stalled.

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Tue, Aug 7, 2018 at 10:45 AM Tamas K Lengyel
<tamas.k.lengyel@gmail.com> wrote:
>
> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
> <tamas.k.lengyel@gmail.com> wrote:
> >
> > On Tue, Aug 7, 2018 at 9:09 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> > >
> > > On Tue, Aug 07, 2018 at 08:45:07AM -0600, Tamas K Lengyel wrote:
> > > > On Tue, Aug 7, 2018 at 8:37 AM Roger Pau Monné <roger.pau@citrix.com> wrote:
> > > > >
> > > > > On Tue, Aug 07, 2018 at 08:29:49AM -0600, Tamas K Lengyel wrote:
> > > > > > On Tue, Aug 7, 2018 at 8:04 AM Roger Pau Monne <roger.pau@citrix.com> wrote:
> > > > > > >
> > > > > > > Hello,
> > > > > > >
> > > > > > > The following series implement a workaround for missing RMRR
> > > > > > > entries for a PVH Dom0. It's based on the iommu_inclusive_mapping VTd
> > > > > > > option.
> > > > > > >
> > > > > > > The PVH workaround identity maps all regions marked as reserved in the
> > > > > > > memory map.
> > > > > > >
> > > > > > > Note that this workaround is enabled by default on Intel hardware. It's
> > > > > > > also available to AMD hardware, although it's disabled by default in
> > > > > > > that case.
> > > > > > >
> > > > > > > The series can be found at:
> > > > > > >
> > > > > > > git://xenbits.xen.org/people/royger/xen.git iommu_inclusive_v3
> > > > > > >
> > > > > > > Thanks, Roger.
> > > > > > > Roger Pau Monne (4):
> > > > > > >   iommu: introduce dom0-iommu option
> > > > > > >   iommu: make iommu_inclusive_mapping a suboption of dom0-iommu
> > > > > > >   dom0/pvh: change the order of the MMCFG initialization
> > > > > > >   x86/iommu: add reserved dom0-iommu option to map reserved memory
> > > > > > >     ranges
> > > > > > >
> > > > > > >  docs/misc/xen-command-line.markdown         | 47 +++++++++++
> > > > > > >  xen/arch/x86/hvm/dom0_build.c               |  9 ++-
> > > > > > >  xen/arch/x86/hvm/io.c                       |  5 ++
> > > > > > >  xen/arch/x86/x86_64/mm.c                    |  3 +-
> > > > > > >  xen/drivers/passthrough/amd/iommu_init.c    |  2 +-
> > > > > > >  xen/drivers/passthrough/amd/pci_amd_iommu.c | 11 ++-
> > > > > > >  xen/drivers/passthrough/arm/iommu.c         |  4 +
> > > > > > >  xen/drivers/passthrough/iommu.c             | 62 +++++++++++++--
> > > > > > >  xen/drivers/passthrough/vtd/extern.h        |  2 -
> > > > > > >  xen/drivers/passthrough/vtd/iommu.c         | 25 +++---
> > > > > > >  xen/drivers/passthrough/vtd/x86/vtd.c       | 58 +-------------
> > > > > > >  xen/drivers/passthrough/x86/iommu.c         | 87 +++++++++++++++++++++
> > > > > > >  xen/include/asm-x86/hvm/io.h                |  3 +
> > > > > > >  xen/include/xen/iommu.h                     |  8 +-
> > > > > > >  14 files changed, 240 insertions(+), 86 deletions(-)
> > > > > > >
> > > > > > > --
> > > > > >
> > > > > > Hi Roger,
> > > > > > I gave this branch a spin on a Dell XPS laptop booting UEFI with Linux
> > > > > > 4.18-rc8. I was able to get dom0 to boot with PVH but the physical
> > > > > > keyboard of the laptop stopped working, it works no problem with just
> > > > > > Linux 4.18-rc8 or PV dom0, so I had to plug in a USB keyboard. After
> > > > > > running for a minute or two the system starts to slow down to the
> > > > > > point where it becomes unresponsive. The xl dmesg log is filled with
> > > > > > this error:
> > > > > >
> > > > > > (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> > > > > > (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> > > > > > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> > > > > > 4625f3a000, iommu reg = ffff82c00181c000
> > > > > > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> > > > > > (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 4625f3a
> > > > >
> > > > > Is the gmfn always the same (0x4625f3a)?
> > > > >
> > > > > > (XEN)     root_entry[00] = 273a18001
> > > > > > (XEN)     context[10] = 2_27ba35001
> > > > > > (XEN)     l4[000] = 9c0000027ba34107
> > > > > > (XEN)     l3[118] = 8000000000000000
> > > > > > (XEN)     l3[118] not present
> > > > >
> > > > > Can you also paste the full xl dmesg log? I'm specially interested in
> > > > > the memory map of the machine which is printed quite early during Xen
> > > > > boot.
> > > > >
> > > >
> > > > Unfortunately I don't have serial access on this laptop and "xl dmesg"
> > > > gets completely filled with that error so the beginning of the log is
> > > > lost by the time I get a terminal in dom0.
> > >
> > > You can get the memory map while booting in PV mode, it's going to be
> > > exactly the same regardless of whether Dom0 is PV or PVH.
> >
> > This is the PV dmesg:
> >
> > (XEN) Xen version 4.12-unstable (dr@) (gcc (Debian 7.3.0-19) 7.3.0)
> > debug=y  Mon Aug  6 13:42:42 MDT 2018
> > (XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f
> > (XEN) Bootloader: EFI
> > (XEN) Command line: loglvl=all guest_loglvl=all
> > dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null console=vga
> > (XEN) Xen image load base address: 0x5a200000
> > (XEN) Video information:
> > (XEN)  VGA is graphics mode 3200x1800, 32 bpp
> > (XEN) Disc information:
> > (XEN)  Found 0 MBR signatures
> > (XEN)  Found 1 EDD information structures
> > (XEN) EFI RAM map:
> > (XEN)  0000000000000000 - 0000000000058000 (usable)
> > (XEN)  0000000000058000 - 0000000000059000 (reserved)
> > (XEN)  0000000000059000 - 000000000009f000 (usable)
> > (XEN)  000000000009f000 - 0000000000100000 (reserved)
> > (XEN)  0000000000100000 - 000000005f14d000 (usable)
> > (XEN)  000000005f14d000 - 000000005f14e000 (ACPI NVS)
> > (XEN)  000000005f14e000 - 000000005f14f000 (reserved)
> > (XEN)  000000005f14f000 - 000000006ee89000 (usable)
> > (XEN)  000000006ee89000 - 000000006f214000 (reserved)
> > (XEN)  000000006f214000 - 000000006f258000 (ACPI data)
> > (XEN)  000000006f258000 - 000000006f8fd000 (ACPI NVS)
> > (XEN)  000000006f8fd000 - 000000006ffff000 (reserved)
> > (XEN)  000000006ffff000 - 0000000070000000 (usable)
> > (XEN)  0000000070000000 - 0000000078000000 (reserved)
> > (XEN)  0000000078000000 - 0000000078600000 (usable)
> > (XEN)  0000000078600000 - 000000007c800000 (reserved)
> > (XEN)  00000000e0000000 - 00000000f0000000 (reserved)
> > (XEN)  00000000fe000000 - 00000000fe011000 (reserved)
> > (XEN)  00000000fec00000 - 00000000fec01000 (reserved)
> > (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> > (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> > (XEN)  0000000100000000 - 0000000281800000 (usable)
> > (XEN) ACPI: RSDP 6F221000, 0024 (r2 DELL  )
> > (XEN) ACPI: XSDT 6F2210C8, 0114 (r1 DELL    CBX3     1072009 AMI     10013)
> > (XEN) ACPI: FACP 6F246918, 010C (r5 DELL    CBX3     1072009 AMI     10013)
> > (XEN) ACPI: DSDT 6F221270, 256A6 (r2 DELL    CBX3     1072009 INTL 20160422)
> > (XEN) ACPI: FACS 6F8FA180, 0040
> > (XEN) ACPI: APIC 6F246A28, 0084 (r3 DELL    CBX3     1072009 AMI     10013)
> > (XEN) ACPI: FPDT 6F246AB0, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
> > (XEN) ACPI: FIDT 6F246AF8, 00AC (r1 DELL    CBX3     1072009 AMI     10013)
> > (XEN) ACPI: MCFG 6F246BA8, 003C (r1 DELL    CBX3     1072009 MSFT       97)
> > (XEN) ACPI: HPET 6F246BE8, 0038 (r1 DELL    CBX3     1072009 AMI.    5000B)
> > (XEN) ACPI: SSDT 6F246C20, 0359 (r1 SataRe SataTabl     1000 INTL 20160422)
> > (XEN) ACPI: BOOT 6F246F80, 0028 (r1 DELL   CBX3      1072009 AMI     10013)
> > (XEN) ACPI: SSDT 6F246FA8, 12CF (r2 SaSsdt  SaSsdt      3000 INTL 20160422)
> > (XEN) ACPI: HPET 6F248278, 0038 (r1 INTEL   KBL-ULT        1 MSFT       5F)
> > (XEN) ACPI: SSDT 6F2482B0, 0D84 (r2  INTEL xh_rvp07        0 INTL 20160422)
> > (XEN) ACPI: UEFI 6F249038, 0042 (r1                        0             0)
> > (XEN) ACPI: SSDT 6F249080, 0EDE (r2 CpuRef  CpuSsdt     3000 INTL 20160422)
> > (XEN) ACPI: LPIT 6F249F60, 0094 (r1 INTEL   KBL-ULT        0 MSFT       5F)
> > (XEN) ACPI: WSMT 6F249FF8, 0028 (r1 DELL    CBX3           0 MSFT       5F)
> > (XEN) ACPI: SSDT 6F24A020, 0161 (r2 INTEL    HdaDsp        0 INTL 20160422)
> > (XEN) ACPI: SSDT 6F24A188, 029F (r2 INTEL  sensrhub        0 INTL 20160422)
> > (XEN) ACPI: SSDT 6F24A428, 3002 (r2 INTEL  PtidDevc     1000 INTL 20160422)
> > (XEN) ACPI: SSDT 6F24D430, 00DB (r2 INTEL  TbtTypeC        0 INTL 20160422)
> > (XEN) ACPI: DBGP 6F24D510, 0034 (r1 INTEL                  2 MSFT       5F)
> > (XEN) ACPI: DBG2 6F24D548, 0054 (r0 INTEL                  2 MSFT       5F)
> > (XEN) ACPI: SSDT 6F24D5A0, 07AC (r2 INTEL  UsbCTabl     1000 INTL 20160422)
> > (XEN) ACPI: SSDT 6F24DD50, 84F1 (r2 DptfTa DptfTabl     1000 INTL 20160422)
> > (XEN) ACPI: MSDM 6F256248, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
> > (XEN) ACPI: SLIC 6F2562A0, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
> > (XEN) ACPI: NHLT 6F256418, 002D (r0 INTEL  EDK2            2       1000013)
> > (XEN) ACPI: BGRT 6F256448, 0038 (r0     ��           1072009 AMI     10013)
> > (XEN) ACPI: TPM2 6F256480, 0034 (r3        Tpm2Tabl        1 AMI         0)
> > (XEN) ACPI: ASF! 6F2564B8, 00A0 (r32 INTEL       HCG        1 TFSM    F4240)
> > (XEN) ACPI: DMAR 6F256558, 00F0 (r1 INTEL      KBL         1 INTL        1)
> > (XEN) System RAM: 7948MB (8138904kB)
> > (XEN) No NUMA configuration found
> > (XEN) Faking a node at 0000000000000000-0000000281800000
> > (XEN) Domain heap initialised
> > (XEN) vesafb: framebuffer at 0x90000000, mapped to 0xffff82c000201000,
> > using 22528k, total 22528k
> > (XEN) vesafb: mode is 3200x1800x32, linelength=12800, font 8x16
> > (XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
> > (XEN) Couldn't initialize a 3200x1800 framebuffer early.
> > (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 142 (0x8e), Stepping 9
> > (raw 000806e9)
> > (XEN) SMBIOS 3.0 present.
> > (XEN) Using APIC driver default
> > (XEN) ACPI: PM-Timer IO Port: 0x1808 (32 bits)
> > (XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
> > (XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
> > (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
> > (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> > 6f8fa180/0000000000000000, using 32
> > (XEN) ACPI:             wakeup_vec[6f8fa18c], vec_size[20]
> > (XEN) ACPI: Local APIC address 0xfee00000
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
> > (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
> > (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> > (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> > (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
> > (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
> > (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> > (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> > (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> > (XEN) ACPI: IRQ0 used by override.
> > (XEN) ACPI: IRQ2 used by override.
> > (XEN) ACPI: IRQ9 used by override.
> > (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> > (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
> > (XEN) [VT-D]dmar.c:851: Ignore unknown DMAR structure type (0x4)
> > (XEN) [VT-D]dmar.c:851: Ignore unknown DMAR structure type (0x4)
> > (XEN) ERST table was not found
> > (XEN) ACPI: BGRT: invalidating v1 image at 0x6ad04018
> > (XEN) Using ACPI (MADT) for SMP configuration information
> > (XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
> > (XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
> > (XEN) Switched to APIC driver x2apic_cluster.
> > (XEN) xstate: size: 0x440 and states: 0x1f
> > (XEN) mce_intel.c:780: MCA Capability: firstbank 0, extended MCE MSR
> > 0, BCAST, CMCI
> > (XEN) CPU0: Intel machine check reporting enabled
> > (XEN) Speculative mitigation facilities:
> > (XEN)   Hardware features: IBRS/IBPB STIBP
> > (XEN)   Compiled-in support: INDIRECT_THUNK
> > (XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+, Other: IBPB
> > (XEN)   Support for VMs: PV: MSR_SPEC_CTRL RSB EAGER_FPU, HVM:
> > MSR_SPEC_CTRL RSB EAGER_FPU
> > (XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled
> > (XEN) Using scheduler: null Scheduler (null)
> > (XEN) Initializing null scheduler
> > (XEN) WARNING: This is experimental software in development.
> > (XEN) Use at your own risk.
> > (XEN) Platform timer is 23.999MHz HPET
> > (XEN) Detected 2904.041 MHz processor.
> > (XEN) EFI memory map:
> > (XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
> > (XEN)  0000000008000-0000000057fff type=7 attr=000000000000000f
> > (XEN)  0000000058000-0000000058fff type=0 attr=000000000000000f
> > (XEN)  0000000059000-000000005cfff type=7 attr=000000000000000f
> > (XEN)  000000005d000-000000005efff type=2 attr=000000000000000f
> > (XEN)  000000005f000-000000005ffff type=4 attr=000000000000000f
> > (XEN)  0000000060000-000000009efff type=3 attr=000000000000000f
> > (XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
> > (XEN)  0000000100000-000003f225fff type=7 attr=000000000000000f
> > (XEN)  000003f226000-000003f265fff type=4 attr=000000000000000f
> > (XEN)  000003f266000-00000561bbfff type=7 attr=000000000000000f
> > (XEN)  00000561bc000-000005a03afff type=2 attr=000000000000000f
> > (XEN)  000005a03b000-000005b23afff type=1 attr=000000000000000f
> > (XEN)  000005b23b000-000005b3e5fff type=2 attr=000000000000000f
> > (XEN)  000005b3e6000-000005b3e9fff type=7 attr=000000000000000f
> > (XEN)  000005b3ea000-000005b415fff type=2 attr=000000000000000f
> > (XEN)  000005b416000-000005b420fff type=7 attr=000000000000000f
> > (XEN)  000005b421000-000005b458fff type=2 attr=000000000000000f
> > (XEN)  000005b459000-000005b45afff type=7 attr=000000000000000f
> > (XEN)  000005b45b000-000005b462fff type=2 attr=000000000000000f
> > (XEN)  000005b463000-000005b464fff type=7 attr=000000000000000f
> > (XEN)  000005b465000-000005d2e0fff type=2 attr=000000000000000f
> > (XEN)  000005d2e1000-000005d325fff type=1 attr=000000000000000f
> > (XEN)  000005d326000-000005f14cfff type=4 attr=000000000000000f
> > (XEN)  000005f14d000-000005f14dfff type=10 attr=000000000000000f
> > (XEN)  000005f14e000-000005f14efff type=6 attr=800000000000000f
> > (XEN)  000005f14f000-000005f215fff type=4 attr=000000000000000f
> > (XEN)  000005f216000-000005f224fff type=2 attr=000000000000000f
> > (XEN)  000005f225000-000006dfe6fff type=4 attr=000000000000000f
> > (XEN)  000006dfe7000-000006e2fafff type=7 attr=000000000000000f
> > (XEN)  000006e2fb000-000006ee88fff type=3 attr=000000000000000f
> > (XEN)  000006ee89000-000006f213fff type=0 attr=000000000000000f
> > (XEN)  000006f214000-000006f257fff type=9 attr=000000000000000f
> > (XEN)  000006f258000-000006f8fcfff type=10 attr=000000000000000f
> > (XEN)  000006f8fd000-000006ff1ffff type=6 attr=800000000000000f
> > (XEN)  000006ff20000-000006fffefff type=5 attr=800000000000000f
> > (XEN)  000006ffff000-000006fffffff type=4 attr=000000000000000f
> > (XEN)  0000078000000-00000785fffff type=7 attr=000000000000000f
> > (XEN)  0000100000000-00002817fffff type=7 attr=000000000000000f
> > (XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
> > (XEN)  0000070000000-0000077ffffff type=0 attr=070000000000000f
> > (XEN)  0000078600000-000007c7fffff type=0 attr=0000000000000000
> > (XEN)  00000e0000000-00000efffffff type=11 attr=8000000000000000
> > (XEN) Unknown cachability for MFNs 0xe0000-0xeffff
> > (XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
> > (XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
> > (XEN)  00000fee00000-00000fee00fff type=11 attr=800000000000100d
> > (XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d
> > (XEN) Initing memory sharing.
> > (XEN) alt table ffff82d0806739b0 -> ffff82d0806757c4
> > (XEN) PCI: MCFG configuration 0: base e0000000 segment 0000 buses 00 - ff
> > (XEN) PCI: MCFG area at e0000000 reserved in E820
> > (XEN) PCI: Using MCFG for segment 0000 bus 00-ff
> > (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
> > (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
> > (XEN) Intel VT-d Snoop Control not enabled.
> > (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> > (XEN) Intel VT-d Queued Invalidation enabled.
> > (XEN) Intel VT-d Interrupt Remapping enabled.
> > (XEN) Intel VT-d Posted Interrupt not enabled.
> > (XEN) Intel VT-d Shared EPT tables enabled.
> > (XEN) I/O virtualisation enabled
> > (XEN)  - Dom0 mode: Relaxed
> > (XEN) Interrupt remapping enabled
> > (XEN) nr_sockets: 1
> > (XEN) Enabled directed EOI with ioapic_ack_old on!
> > (XEN) ENABLING IO-APIC IRQs
> > (XEN)  -> Using old ACK method
> > (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=0 pin2=0
> > (XEN) TSC deadline timer enabled
> > (XEN) Allocated console ring of 32 KiB.
> > (XEN) mwait-idle: MWAIT substates: 0x11142120
> > (XEN) mwait-idle: v0.4.1 model 0x8e
> > (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> > (XEN) mwait-idle: max C-state count of 8 reached
> > (XEN) VMX: Supported advanced features:
> > (XEN)  - APIC MMIO access virtualisation
> > (XEN)  - APIC TPR shadow
> > (XEN)  - Extended Page Tables (EPT)
> > (XEN)  - Virtual-Processor Identifiers (VPID)
> > (XEN)  - Virtual NMI
> > (XEN)  - MSR direct-access bitmap
> > (XEN)  - Unrestricted Guest
> > (XEN)  - VM Functions
> > (XEN)  - Virtualisation Exceptions
> > (XEN)  - Page Modification Logging
> > (XEN) HVM: ASIDs enabled.
> > (XEN) HVM: VMX enabled
> > (XEN) HVM: Hardware Assisted Paging (HAP) detected
> > (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> > (XEN) mwait-idle: max C-state count of 8 reached
> > (XEN) mwait-idle: max C-state count of 8 reached
> > (XEN) mwait-idle: max C-state count of 8 reached
> > (XEN) Brought up 4 CPUs
> > (XEN) build-id: 1e3eb01a83018530b8a66809744145e8
> > (XEN) Running stub recovery selftests...
> > (XEN) traps.c:1570: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] ->
> > ffff82d08037f3f2
> > (XEN) traps.c:755: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] ->
> > ffff82d08037f3f2
> > (XEN) traps.c:1097: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] ->
> > ffff82d08037f3f2
> > (XEN) ACPI sleep modes: S3
> > (XEN) VPMU: disabled
> > (XEN) mcheck_poll: Machine check polling timer started.
> > (XEN) Dom0 has maximum 504 PIRQs
> > (XEN) sched_null.c:347: 0 <-- d0v0
> > (XEN) NX (Execute Disable) protection active
> > (XEN) *** Building a PV Dom0 ***
> > (XEN) ELF: phdr: paddr=0x1000000 memsz=0x123c000
> > (XEN) ELF: phdr: paddr=0x2400000 memsz=0x2a7000
> > (XEN) ELF: phdr: paddr=0x26a7000 memsz=0x25000
> > (XEN) ELF: phdr: paddr=0x26cc000 memsz=0x4ba000
> > (XEN) ELF: memory: 0x1000000 -> 0x2b86000
> > (XEN) ELF: note: GUEST_OS = "linux"
> > (XEN) ELF: note: GUEST_VERSION = "2.6"
> > (XEN) ELF: note: XEN_VERSION = "xen-3.0"
> > (XEN) ELF: note: VIRT_BASE = 0xffffffff80000000
> > (XEN) ELF: note: INIT_P2M = 0x8000000000
> > (XEN) ELF: note: ENTRY = 0xffffffff826cc180
> > (XEN) ELF: note: HYPERCALL_PAGE = 0xffffffff81001000
> > (XEN) ELF: note: FEATURES = "!writable_page_tables|pae_pgdir_above_4gb"
> > (XEN) ELF: note: SUPPORTED_FEATURES = 0x8801
> > (XEN) ELF: note: PAE_MODE = "yes"
> > (XEN) ELF: note: LOADER = "generic"
> > (XEN) ELF: note: unknown (0xd)
> > (XEN) ELF: note: SUSPEND_CANCEL = 0x1
> > (XEN) ELF: note: MOD_START_PFN = 0x1
> > (XEN) ELF: note: HV_START_LOW = 0xffff800000000000
> > (XEN) ELF: note: PADDR_OFFSET = 0
> > (XEN) ELF: note: PHYS32_ENTRY = 0x1000380
> > (XEN) ELF: Found PVH image
> > (XEN) ELF: addresses:
> > (XEN)     virt_base        = 0xffffffff80000000
> > (XEN)     elf_paddr_offset = 0x0
> > (XEN)     virt_offset      = 0xffffffff80000000
> > (XEN)     virt_kstart      = 0xffffffff81000000
> > (XEN)     virt_kend        = 0xffffffff82b86000
> > (XEN)     virt_entry       = 0xffffffff826cc180
> > (XEN)     p2m_base         = 0x8000000000
> > (XEN)  Xen  kernel: 64-bit, lsb, compat32
> > (XEN)  Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2b86000
> > (XEN) PHYSICAL MEMORY ARRANGEMENT:
> > (XEN)  Dom0 alloc.:   0000000270000000->0000000274000000 (1018277
> > pages to be allocated)
> > (XEN)  Init. ramdisk: 000000027e1a5000->00000002817ff72a
> > (XEN) VIRTUAL MEMORY ARRANGEMENT:
> > (XEN)  Loaded kernel: ffffffff81000000->ffffffff82b86000
> > (XEN)  Init. ramdisk: 0000000000000000->0000000000000000
> > (XEN)  Phys-Mach map: 0000008000000000->0000008000800000
> > (XEN)  Start info:    ffffffff82b86000->ffffffff82b864b4
> > (XEN)  Xenstore ring: 0000000000000000->0000000000000000
> > (XEN)  Console ring:  0000000000000000->0000000000000000
> > (XEN)  Page tables:   ffffffff82b87000->ffffffff82ba2000
> > (XEN)  Boot stack:    ffffffff82ba2000->ffffffff82ba3000
> > (XEN)  TOTAL:         ffffffff80000000->ffffffff83000000
> > (XEN)  ENTRY ADDRESS: ffffffff826cc180
> > (XEN) Dom0 has maximum 2 VCPUs
> > (XEN) sched_null.c:347: 1 <-- d0v1
> > (XEN) ELF: phdr 0 at 0xffffffff81000000 -> 0xffffffff8223c000
> > (XEN) ELF: phdr 1 at 0xffffffff82400000 -> 0xffffffff826a7000
> > (XEN) ELF: phdr 2 at 0xffffffff826a7000 -> 0xffffffff826cc000
> > (XEN) ELF: phdr 3 at 0xffffffff826cc000 -> 0xffffffff82911000
> > (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> > 7b380000, iommu reg = ffff82c00181c000
> > (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> > (XEN) Initial low memory virq threshold set at 0x4000 pages.
> > (XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
> > (XEN) .........................................done.
> > (XEN) Std. Loglevel: All
> > (XEN) Guest Loglevel: All
> > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
> > input to Xen)
> > (XEN) Freed 2048kB init memory
> > (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 0000001b from
> > 0x00000000fee00d00 to 0x00000000fee00100
> > (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 0000001b from
> > 0x00000000fee00d00 to 0x00000000fee00900
> > (XEN) emul-priv-op.c:1166:d0v1 Domain attempted WRMSR 0000001b from
> > 0x00000000fee00c00 to 0x00000000fee00000
> > (XEN) emul-priv-op.c:1166:d0v1 Domain attempted WRMSR 0000001b from
> > 0x00000000fee00c00 to 0x00000000fee00800
> > (XEN) d0: Forcing write emulation on MFNs e0000-effff
> > (XEN) PCI add device 0000:00:00.0
> > (XEN) PCI add device 0000:00:02.0
> > (XEN) PCI add device 0000:00:04.0
> > (XEN) PCI add device 0000:00:14.0
> > (XEN) PCI add device 0000:00:14.2
> > (XEN) PCI add device 0000:00:15.0
> > (XEN) PCI add device 0000:00:15.1
> > (XEN) PCI add device 0000:00:16.0
> > (XEN) PCI add device 0000:00:1c.0
> > (XEN) PCI add device 0000:00:1c.4
> > (XEN) PCI add device 0000:00:1c.5
> > (XEN) PCI add device 0000:00:1d.0
> > (XEN) PCI add device 0000:00:1f.0
> > (XEN) PCI add device 0000:00:1f.2
> > (XEN) PCI add device 0000:00:1f.3
> > (XEN) PCI add device 0000:00:1f.4
> > (XEN) PCI add device 0000:3a:00.0
> > (XEN) PCI add device 0000:3b:00.0
> > (XEN) PCI add device 0000:3c:00.0
> > (XEN) d0: Forcing read-only access to MFN fed00
> > (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> > [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> > (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> > [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> > (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> > [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> > (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> > [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> > (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> > 0x004280e800dd8088 to 0x004280e8009d8088
> > (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> > 0x004280e800dd8088 to 0x004280e800dd0088
> > (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> > 0x004280e800dd8088 to 0x004280e800dc8088
> > (XEN) emul-priv-op.c:1166:d0v0 Domain attempted WRMSR 00000610 from
> > 0x004280e800dd8088 to 0x004200e800dd8088
> > (XEN) traps.c:1570: GPF (0000): ffff82d080371462
> > [emul-priv-op.c#read_msr+0x3a2/0x47c] -> ffff82d08037fbae
> > (XEN) PCI add device 0000:01:00.0
> > (XEN) PCI add device 0000:02:00.0
> > (XEN) PCI add device 0000:02:01.0
> > (XEN) PCI add device 0000:02:02.0
> > (XEN) PCI add device 0000:03:00.0
> > (XEN) PCI add device 0000:39:00.0
> > (XEN) PCI remove device 0000:03:00.0
> > (XEN) PCI remove device 0000:02:00.0
> > (XEN) PCI remove device 0000:02:01.0
> > (XEN) PCI remove device 0000:39:00.0
> > (XEN) PCI remove device 0000:02:02.0
> > (XEN) PCI remove device 0000:01:00.0
> > (XEN) PCI add device 0000:01:00.0
> > (XEN) PCI add device 0000:02:00.0
> > (XEN) PCI add device 0000:02:01.0
> > (XEN) PCI add device 0000:02:02.0
> > (XEN) PCI add device 0000:03:00.0
> > (XEN) PCI add device 0000:39:00.0
>
>
> This is the log on another Dell laptop with which I have serial access
> and it has the same error, physical keyboard/touchpad doesn't work but
> attaching one via USB does:
>
> (XEN) Xen version 4.12-unstable (drt@) (gcc (Ubuntu 7.3.0-16ubuntu3)
> 7.3.0) debug=y  Mon Aug  6 16:40:02 MDT 2018
> (XEN) Latest ChangeSet: Fri Aug 3 10:01:36 2018 +0200 git:ddba1c2b1f
> (XEN) Bootloader: EFI
> (XEN) Command line: loglvl=all guest_loglvl=all
> dom0_mem=4096M,max:4096M dom0_max_vcpus=2 sched=null dom0=pvh
> iommu=required,debug dom0-iommu=relaxed console=vga,com1 com1=1152
> 00,8n1,pci
> (XEN) Xen image load base address: 0x99200000
> (XEN) Video information:
> (XEN)  VGA is graphics mode 1920x1080, 32 bpp
> (XEN) Disc information:
> (XEN)  Found 0 MBR signatures
> (XEN)  Found 1 EDD information structures
> (XEN) EFI RAM map:
> (XEN)  0000000000000000 - 0000000000058000 (usable)
> (XEN)  0000000000058000 - 0000000000059000 (reserved)
> (XEN)  0000000000059000 - 000000000009f000 (usable)
> (XEN)  000000000009f000 - 0000000000100000 (reserved)
> (XEN)  0000000000100000 - 000000009cba2000 (usable)
> (XEN)  000000009cba2000 - 000000009cba3000 (ACPI NVS)
> (XEN)  000000009cba3000 - 000000009cbcd000 (reserved)
> (XEN)  000000009cbcd000 - 00000000ab080000 (usable)
> (XEN)  00000000ab080000 - 00000000ab3f5000 (reserved)
> (XEN)  00000000ab3f5000 - 00000000ab42e000 (ACPI data)
> (XEN)  00000000ab42e000 - 00000000aba6c000 (ACPI NVS)
> (XEN)  00000000aba6c000 - 00000000ac0ff000 (reserved)
> (XEN)  00000000ac0ff000 - 00000000ac100000 (usable)
> (XEN)  00000000ac100000 - 00000000b0000000 (reserved)
> (XEN)  00000000f0000000 - 00000000f8000000 (reserved)
> (XEN)  00000000fe000000 - 00000000fe011000 (reserved)
> (XEN)  00000000fec00000 - 00000000fec01000 (reserved)
> (XEN)  00000000fed20000 - 00000000fed80000 (reserved)
> (XEN)  00000000fee00000 - 00000000fee01000 (reserved)
> (XEN)  00000000ff000000 - 0000000100000000 (reserved)
> (XEN)  0000000100000000 - 000000044e000000 (usable)
> (XEN) ACPI: RSDP AB400000, 0024 (r2 DELL  )
> (XEN) ACPI: XSDT AB4000B0, 00E4 (r1 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: FACP AB41FB78, 010C (r5 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: DSDT AB400228, 1F94F (r2 DELL    CBX3     1072009 INTL 20120913)
> (XEN) ACPI: FACS ABA69E80, 0040
> (XEN) ACPI: UEFI AB420000, 063A (r1  INTEL RstSataE        0   ��        0)
> (XEN) ACPI: APIC AB420640, 0084 (r3 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: FPDT AB4206C8, 0044 (r1 DELL    CBX3     1072009 AMI     10013)
> (XEN) ACPI: MCFG AB420710, 003C (r1 DELL    CBX3     1072009 MSFT       97)
> (XEN) ACPI: HPET AB420750, 0038 (r1 DELL    CBX3     1072009 AMI.    5000B)
> (XEN) ACPI: SSDT AB420788, 046D (r1 SataRe SataTabl     1000 INTL 20120913)
> (XEN) ACPI: LPIT AB420BF8, 0094 (r1 INTEL   SKL-ULT        0 MSFT       5F)
> (XEN) ACPI: SSDT AB420C90, 0248 (r2 INTEL  sensrhub        0 INTL 20120913)
> (XEN) ACPI: SSDT AB420ED8, 2BAE (r2 INTEL  PtidDevc     1000 INTL 20120913)
> (XEN) ACPI: SSDT AB423A88, 0814 (r2 INTEL  xh_rvp07        0 INTL 20120913)
> (XEN) ACPI: DBGP AB4242A0, 0034 (r1 INTEL                  0 MSFT       5F)
> (XEN) ACPI: DBG2 AB4242D8, 0054 (r0 INTEL                  0 MSFT       5F)
> (XEN) ACPI: SSDT AB424330, 35DA (r2 SaSsdt  SaSsdt      3000 INTL 20120913)
> (XEN) ACPI: UEFI AB427910, 0042 (r1                        0             0)
> (XEN) ACPI: SSDT AB427958, 0E73 (r2 CpuRef  CpuSsdt     3000 INTL 20120913)
> (XEN) ACPI: SSDT AB4287D0, 3BFD (r2 DptfTa DptfTabl     1000 INTL 20120913)
> (XEN) ACPI: MSDM AB42C3D0, 0055 (r3 DELL    CBX3     6222004 AMI     10013)
> (XEN) ACPI: SLIC AB42C428, 0176 (r3 DELL    CBX3     1072009 MSFT    10013)
> (XEN) ACPI: TCPA AB42C5A0, 0032 (r2 ALASKA  NAPAASF        0 MSFT  1000013)
> (XEN) ACPI: DMAR AB42C5D8, 00A8 (r1 INTEL      SKL         1 INTL        1)
> (XEN) ACPI: UEFI AB42C680, 005C (r1  INTEL RstSataV        0   ��        0)
> (XEN) ACPI: ASF! AB42C6E0, 00A5 (r32 INTEL       HCG        1 TFSM    F4240)
> (XEN) ACPI: BGRT AB42C788, 0038 (r0     ��           1072009 AMI     10013)
> (XEN) System RAM: 16271MB (16662480kB)
> (XEN) No NUMA configuration found
> (XEN) Faking a node at 0000000000000000-000000044e000000
> (XEN) Domain heap initialised
> (XEN) vesafb: framebuffer at 0xd0000000, mapped to 0xffff82c000201000,
> using 8128k, total 8128k
> (XEN) vesafb: mode is 1920x1080x32, linelength=7680, font 8x16
> (XEN) vesafb: Truecolor: size=8:8:8:8, shift=24:16:8:0
> (XEN) Couldn't initialize a 1920x1080 framebuffer early.
> (XEN) CPU Vendor: Intel, Family 6 (0x6), Model 78 (0x4e), Stepping 3
> (raw 000406e3)
> (XEN) SMBIOS 3.0 present.
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1808 (32 bits)
> (XEN) ACPI: v5 SLEEP INFO: control[1:1804], status[1:1800]
> (XEN) ACPI: Invalid sleep control/status register data: 0:0x8:0x3 0:0x8:0x3
> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1804,1:0], pm1x_evt[1:1800,1:0]
> (XEN) ACPI: 32/64X FACS address mismatch in FADT -
> aba69e80/0000000000000000, using 32
> (XEN) ACPI:             wakeup_vec[aba69e8c], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x01] enabled)
> (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x03] enabled)
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 2, version 32, address 0xfec00000, GSI 0-119
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) ACPI: IRQ9 used by override.
> (XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
> (XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
> (XEN) [VT-D]Host address width 39
> (XEN) [VT-D]found ACPI_DMAR_DRHD:
> (XEN) [VT-D]  dmaru->address = fed90000
> (XEN) [VT-D]drhd->address = fed90000 iommu->reg = ffff82c000a0c000
> (XEN) [VT-D]cap = 1c0000c40660462 ecap = 7e3ff0505e
> (XEN) [VT-D] endpoint: 0000:00:02.0
> (XEN) [VT-D]found ACPI_DMAR_DRHD:
> (XEN) [VT-D]  dmaru->address = fed91000
> (XEN) [VT-D]drhd->address = fed91000 iommu->reg = ffff82c000a0e000
> (XEN) [VT-D]cap = d2008c40660462 ecap = f050da
> (XEN) [VT-D] IOAPIC: 0000:f0:1f.0
> (XEN) [VT-D] MSI HPET: 0000:00:1f.0
> (XEN) [VT-D]  flags: INCLUDE_ALL
> (XEN) [VT-D]found ACPI_DMAR_RMRR:
> (XEN) [VT-D] endpoint: 0000:00:14.0
> (XEN) [VT-D]dmar.c:638:   RMRR region: base_addr ab198000 end_addr ab1b7fff
> (XEN) [VT-D]found ACPI_DMAR_RMRR:
> (XEN) [VT-D] endpoint: 0000:00:02.0
> (XEN) [VT-D]dmar.c:638:   RMRR region: base_addr ad800000 end_addr afffffff
> (XEN) ERST table was not found
> (XEN) ACPI: BGRT: invalidating v1 image at 0xa837d018
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
> (XEN) IRQ limits: 120 GSI, 840 MSI/MSI-X
> (XEN) Not enabling x2APIC (upon firmware request)
> (XEN) xstate: size: 0x440 and states: 0x1f
> (XEN) mce_intel.c:780: MCA Capability: firstbank 0, extended MCE MSR
> 0, BCAST, CMCI
> (XEN) CPU0: Intel machine check reporting enabled
> (XEN) Speculative mitigation facilities:
> (XEN)   Hardware features: IBRS/IBPB STIBP
> (XEN)   Compiled-in support: INDIRECT_THUNK
> (XEN)   Xen settings: BTI-Thunk JMP, SPEC_CTRL: IBRS+, Other: IBPB
> (XEN)   Support for VMs: PV: MSR_SPEC_CTRL RSB EAGER_FPU, HVM:
> MSR_SPEC_CTRL RSB EAGER_FPU
> (XEN)   XPTI (64-bit PV only): Dom0 enabled, DomU enabled
> (XEN) Using scheduler: null Scheduler (null)
> (XEN) Initializing null scheduler
> (XEN) WARNING: This is experimental software in development.
> (XEN) Use at your own risk.
> (XEN) Platform timer is 23.999MHz HPET
> (XEN) Detected 2808.069 MHz processor.
> (XEN) EFI memory map:
> (XEN)  0000000000000-0000000007fff type=3 attr=000000000000000f
> (XEN)  0000000008000-0000000057fff type=7 attr=000000000000000f
> (XEN)  0000000058000-0000000058fff type=0 attr=000000000000000f
> (XEN)  0000000059000-000000005cfff type=7 attr=000000000000000f
> (XEN)  000000005d000-000000005efff type=2 attr=000000000000000f
> (XEN)  000000005f000-000000005ffff type=4 attr=000000000000000f
> (XEN)  0000000060000-000000009efff type=3 attr=000000000000000f
> (XEN)  000000009f000-000000009ffff type=0 attr=000000000000000f
> (XEN)  0000000100000-000007cc3cfff type=7 attr=000000000000000f
> (XEN)  000007cc3d000-000007cc7cfff type=4 attr=000000000000000f
> (XEN)  000007cc7d000-0000095330fff type=7 attr=000000000000000f
> (XEN)  0000095331000-00000990c7fff type=2 attr=000000000000000f
> (XEN)  00000990c8000-000009a3b0fff type=1 attr=000000000000000f
> (XEN)  000009a3b1000-000009a407fff type=2 attr=000000000000000f
> (XEN)  000009a408000-000009a408fff type=7 attr=000000000000000f
> (XEN)  000009a409000-000009a431fff type=2 attr=000000000000000f
> (XEN)  000009a432000-000009a432fff type=7 attr=000000000000000f
> (XEN)  000009a433000-000009a440fff type=2 attr=000000000000000f
> (XEN)  000009a441000-000009a442fff type=7 attr=000000000000000f
> (XEN)  000009a443000-000009a44dfff type=2 attr=000000000000000f
> (XEN)  000009a44e000-000009a44ffff type=7 attr=000000000000000f
> (XEN)  000009a450000-000009b062fff type=2 attr=000000000000000f
> (XEN)  000009b063000-000009b0a7fff type=1 attr=000000000000000f
> (XEN)  000009b0a8000-000009cba1fff type=4 attr=000000000000000f
> (XEN)  000009cba2000-000009cba2fff type=10 attr=000000000000000f
> (XEN)  000009cba3000-000009cbccfff type=6 attr=800000000000000f
> (XEN)  000009cbcd000-000009cc2cfff type=4 attr=000000000000000f
> (XEN)  000009cc2d000-000009cc3bfff type=2 attr=000000000000000f
> (XEN)  000009cc3c000-00000aa767fff type=4 attr=000000000000000f
> (XEN)  00000aa768000-00000aa97ffff type=7 attr=000000000000000f
> (XEN)  00000aa980000-00000ab07ffff type=3 attr=000000000000000f
> (XEN)  00000ab080000-00000ab3f4fff type=0 attr=000000000000000f
> (XEN)  00000ab3f5000-00000ab42dfff type=9 attr=000000000000000f
> (XEN)  00000ab42e000-00000aba6bfff type=10 attr=000000000000000f
> (XEN)  00000aba6c000-00000ac026fff type=6 attr=800000000000000f
> (XEN)  00000ac027000-00000ac0fefff type=5 attr=800000000000000f
> (XEN)  00000ac0ff000-00000ac0fffff type=4 attr=000000000000000f
> (XEN)  0000100000000-000044dffffff type=7 attr=000000000000000f
> (XEN)  00000000a0000-00000000fffff type=0 attr=0000000000000000
> (XEN)  00000ac100000-00000afffffff type=0 attr=0000000000000000
> (XEN)  00000f0000000-00000f7ffffff type=11 attr=800000000000100d
> (XEN)  00000fe000000-00000fe010fff type=11 attr=8000000000000001
> (XEN)  00000fec00000-00000fec00fff type=11 attr=8000000000000001
> (XEN)  00000fed20000-00000fed7ffff type=0 attr=0000000000000000
> (XEN)  00000fee00000-00000fee00fff type=11 attr=800000000000100d
> (XEN)  00000ff000000-00000ffffffff type=11 attr=800000000000100d
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d080672850 -> ffff82d080674656
> (XEN) spurious 8259A interrupt: IRQ7.
> (XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
> (XEN) PCI: MCFG area at f0000000 reserved in E820
> (XEN) PCI: Using MCFG for segment 0000 bus 00-7f
> (XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) Intel VT-d Snoop Control not enabled.
> (XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
> (XEN) Intel VT-d Queued Invalidation enabled.
> (XEN) Intel VT-d Interrupt Remapping enabled.
> (XEN) Intel VT-d Posted Interrupt not enabled.
> (XEN) Intel VT-d Shared EPT tables enabled.
> (XEN) I/O virtualisation enabled
> (XEN)  - Dom0 mode: Relaxed
> (XEN) Interrupt remapping enabled
> (XEN) nr_sockets: 1
> (XEN) Enabled directed EOI with ioapic_ack_old on!
> (XEN) ENABLING IO-APIC IRQs
> (XEN)  -> Using old ACK method
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=0 pin2=0
> (XEN) TSC deadline timer enabled
> (XEN) Allocated console ring of 32 KiB.
> (XEN) mwait-idle: MWAIT substates: 0x11142120
> (XEN) mwait-idle: v0.4.1 model 0x4e
> (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) VMX: Supported advanced features:
> (XEN)  - APIC MMIO access virtualisation
> (XEN)  - APIC TPR shadow
> (XEN)  - Extended Page Tables (EPT)
> (XEN)  - Virtual-Processor Identifiers (VPID)
> (XEN)  - Virtual NMI
> (XEN)  - MSR direct-access bitmap
> (XEN)  - Unrestricted Guest
> (XEN)  - VMCS shadowing
> (XEN)  - VM Functions
> (XEN)  - Virtualisation Exceptions
> (XEN)  - Page Modification Logging
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) detected
> (XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) mwait-idle: max C-state count of 8 reached
> (XEN) Brought up 4 CPUs
> (XEN) build-id: 8b75b598bb19e50dd6b62cdd229ab5d3
> (XEN) Running stub recovery selftests...
> (XEN) traps.c:1570: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] ->
> ffff82d08037d3f2
> (XEN) traps.c:755: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] ->
> ffff82d08037d3f2
> (XEN) traps.c:1097: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] ->
> ffff82d08037d3f2
> (XEN) ACPI sleep modes: S3
> (XEN) VPMU: disabled
> (XEN) mcheck_poll: Machine check polling timer started.
> (XEN) Dom0 has maximum 504 PIRQs
> (XEN) sched_null.c:347: 0 <-- d0v0
> (XEN) NX (Execute Disable) protection active
> (XEN) *** Building a PVH Dom0 ***
> (XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map
> (XEN) Bogus DMIBAR 0xfed18001 on 0000:00:00.0
> (XEN) [VT-D]d0:PCIe: map 0000:00:02.0
> (XEN) [VT-D]d0:PCI: map 0000:00:04.0
> (XEN) [VT-D]d0:PCI: map 0000:00:14.0
> (XEN) [VT-D]d0:PCI: map 0000:00:14.2
> (XEN) [VT-D]d0:PCI: map 0000:00:16.0
> (XEN) [VT-D]d0:PCI: map 0000:00:16.3
> (XEN) [VT-D]d0:PCI: map 0000:00:17.0
> (XEN) [VT-D]d0:PCI: map 0000:00:1f.0
> (XEN) [VT-D]d0:PCI: map 0000:00:1f.2
> (XEN) [VT-D]d0:PCI: map 0000:00:1f.3
> (XEN) [VT-D]d0:PCI: map 0000:00:1f.4
> (XEN) [VT-D]d0:PCI: map 0000:00:1f.6
> (XEN) [VT-D]d0:PCIe: map 0000:01:00.0
> (XEN) [VT-D]d0:PCIe: map 0000:02:00.0
> (XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c000a0c000
> (XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c000a0e000
> (XEN) sched_null.c:347: 1 <-- d0v1
> (XEN) WARNING: PVH is an experimental mode with limited functionality
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Scrubbing Free RAM on 1 nodes using 2 CPUs
> (XEN) .....................................................................done.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
> input to Xen)
> (XEN) Freed 2048kB init memory
> (XEN) Dom0 callback via changed to Direct Vector 0xf3
> [    0.000000] Xen Platform PCI: unrecognised magic value
> (XEN) d0: bind: m_gsi=9 g_gsi=9
> [    0.127065] pci 0000:00:00.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.136467] pci 0000:00:02.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.137007] pci 0000:00:04.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.139389] pci 0000:00:14.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.140103] pci 0000:00:14.2: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.140632] pci 0000:00:16.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.141133] pci 0000:00:16.3: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.142337] pci 0000:00:17.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.143120] pci 0000:00:1c.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.143866] pci 0000:00:1d.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.144407] pci 0000:00:1f.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.145467] pci 0000:00:1f.2: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.147911] pci 0000:00:1f.3: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.148648] pci 0000:00:1f.4: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.152896] pci 0000:00:1f.6: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.165264] pci 0000:01:00.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> [    0.169878] pci 0000:02:00.0: Failed to add - passthrough or
> MSI/MSI-X might fail!
> ....
> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> 428f926000, iommu reg = ffff82c000a0c000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
> (XEN)     root_entry[00] = 43aaae001
> (XEN)     context[10] = 2_43cf92001
> (XEN)     l4[000] = 9c0000043cf91107
> (XEN)     l3[10a] = 8000000000000000
> (XEN)     l3[10a] not present
>
> The fault is repeated a million times per second and the system is
> pretty much stalled.
>

Interestingly, PV dom0 also has issues with this device:

(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.597359] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.597393] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.613990] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.614080] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.630322] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.630359] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.647397] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.647439] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.664462] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.664498] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.681062] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.681099] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.697740] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.697775] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.712564] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
[   43.712601] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
6c6ddf6000, iommu reg = ffff82c000a0c000
(XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
[   43.828642] [drm:gen8_de_irq_handler [i915]] *ERROR* Fault errors
on pipe A: 0x00000080
(XEN) Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1173
(XEN) ----[ Xen-4.12-unstable  x86_64  debug=y   Not tainted ]----
(XEN) CPU:    0
(XEN) RIP:    e008:[<ffff82d0802834dc>] do_IRQ+0x4cb/0x6a9
(XEN) RFLAGS: 0000000000010087   CONTEXT: hypervisor (d0v0)
(XEN) rax: ffff82d08095f440   rbx: ffff830099a27ef8   rcx: 0000000000000001
(XEN) rdx: 0000000000000001   rsi: 0000000000000031   rdi: 0000000000000041
(XEN) rbp: ffff830099a27ee8   rsp: ffff830099a27e88   r8:  ffff830099a27fff
(XEN) r9:  0000000000000031   r10: 0000000000000000   r11: 0000000000000000
(XEN) r12: ffff83044843dd60   r13: 0000000000000031   r14: ffff830440408000
(XEN) r15: 0000000000000080   cr0: 0000000080050033   cr4: 0000000000372660
(XEN) cr3: 000000042f6ad000   cr2: 0000560418b9f000
(XEN) fsb: 00007f8a5f4e4a80   gsb: ffff880154e00000   gss: 0000000000000000
(XEN) ds: 0000   es: 0000   fs: 0000   gs: 0000   ss: 0000   cs: e008
(XEN) Xen code around <ffff82d0802834dc> (do_IRQ+0x4cb/0x6a9):
(XEN)  0f 8e 68 ff ff ff 0f 0b <0f> 0b 0f 0b 0f 0b b8 00 00 00 00 eb 4f 83 bb 1c
(XEN) Xen stack trace from rsp=ffff830099a27e88:
(XEN)    ffff82d000000000 ffff830440408024 0000000000000000 0000000000000080
(XEN)    ffff82d08037b841 ffff82d08037b835 ffff82d08037b841 0000000000000000
(XEN)    0000000000000000 0000000000000000 ffff830099a27fff 0000000000000000
(XEN)    00007cff665d80e7 ffff82d08037b8aa 0000000000000000 0000000000000000
(XEN)    ffff880154e1ff9c ffff88015093c000 ffff880154e03f98 000000000000001c
(XEN)    ffffffff829827ed 0000000000000040 0000000000000004 ffffc90041473f58
(XEN)    0000000000000080 00000000d436f61e ffff880144e4ea00 0000000000000001
(XEN)    ffff880144e4eaa4 0000003100000000 ffffffff815ef436 000000000000e033
(XEN)    0000000000000246 ffff880154e03f50 000000000000e02b 0000000000000000
(XEN)    0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN)    ffff83009c9fc000 0000000000000000 0000000000372660 0000000000000000
(XEN)    8000000440446002 0000060100000000 0000000000000000
(XEN) Xen call trace:
(XEN)    [<ffff82d0802834dc>] do_IRQ+0x4cb/0x6a9
(XEN)    [<ffff82d08037b8aa>] common_interrupt+0x10a/0x120
(XEN)
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 0:
(XEN) Assertion '(sp == 0) || (peoi[sp-1].vector < vector)' failed at irq.c:1173
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Roger Pau Monné]

On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
> <tamas.k.lengyel@gmail.com> wrote:
> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> 428f926000, iommu reg = ffff82c000a0c000
> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
> (XEN)     root_entry[00] = 43aaae001
> (XEN)     context[10] = 2_43cf92001
> (XEN)     l4[000] = 9c0000043cf91107
> (XEN)     l3[10a] = 8000000000000000
> (XEN)     l3[10a] not present
> 
> The fault is repeated a million times per second and the system is
> pretty much stalled.

As Jan says, this page is outside of any range in the memory map. I
wonder however what's in there.

I think (also seeing the PV issues) you should bring this up with the
driver maintainers, it might actually be a bug that the driver is
trying to access such address.

In the meantime, you can try to add to the command line:

rmrr=0x428f926=0:0:2.0

In order to force an iommu mapping of this address.

Thanks, Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

[Roger Pau Monné]

On Tue, Aug 07, 2018 at 04:02:43PM +0200, Roger Pau Monne wrote:
> @@ -149,36 +204,9 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>      for ( i = 0; i < top; i++ )
>      {
>          unsigned long pfn = pdx_to_pfn(i);
> -        bool map;
>          int rc;
>  
> -        /*
> -         * Set up 1:1 mapping for dom0. Default to include only
> -         * conventional RAM areas and let RMRRs include needed reserved
> -         * regions. When set, the inclusive mapping additionally maps in
> -         * every pfn up to 4GB except those that fall in unusable ranges.
> -         */
> -        if ( pfn > max_pfn && !mfn_valid(_mfn(pfn)) )
> -            continue;
> -
> -        if ( iommu_dom0_inclusive && pfn <= max_pfn )
> -            map = !page_is_ram_type(pfn, RAM_TYPE_UNUSABLE);
> -        else
> -            map = page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL);
> -
> -        if ( !map )
> -            continue;
> -
> -        /* Exclude Xen bits */
> -        if ( xen_in_range(pfn) )
> -            continue;
> -
> -        /*
> -         * If dom0-strict mode is enabled then exclude conventional RAM
> -         * and let the common code map dom0's pages.
> -         */
> -        if ( iommu_dom0_strict &&
> -             page_is_ram_type(pfn, RAM_TYPE_CONVENTIONAL) )
> +        if ( !hwdom_iommu_map(d, pfn, max_pfn) )
>              continue;
>  
>          rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);

I've lost a chunk here during one of the rebases, so the following
diff should be added to the patch in order to create maps if the iommu
page tables are shared:

diff --git a/xen/drivers/passthrough/x86/iommu.c b/xen/drivers/passthrough/x86/iommu.c
index 6aec43ed1a..6271d8b671 100644
--- a/xen/drivers/passthrough/x86/iommu.c
+++ b/xen/drivers/passthrough/x86/iommu.c
@@ -209,7 +209,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
         if ( !hwdom_iommu_map(d, pfn, max_pfn) )
             continue;
 
-        rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
+        if ( iommu_use_hap_pt(d) )
+        {
+            ASSERT(is_hvm_domain(d));
+            rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
+        }
+        else
+            rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
         if ( rc )
             printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
                    d->domain_id, rc);

I can resend the series in order to ease review.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Jan Beulich]

>>> On 08.08.18 at 10:25, <roger.pau@citrix.com> wrote:
> On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
>> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
>> <tamas.k.lengyel@gmail.com> wrote:
>> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
>> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
>> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
>> 428f926000, iommu reg = ffff82c000a0c000
>> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
>> (XEN)     root_entry[00] = 43aaae001
>> (XEN)     context[10] = 2_43cf92001
>> (XEN)     l4[000] = 9c0000043cf91107
>> (XEN)     l3[10a] = 8000000000000000
>> (XEN)     l3[10a] not present
>> 
>> The fault is repeated a million times per second and the system is
>> pretty much stalled.
> 
> As Jan says, this page is outside of any range in the memory map. I
> wonder however what's in there.
> 
> I think (also seeing the PV issues) you should bring this up with the
> driver maintainers, it might actually be a bug that the driver is
> trying to access such address.

Right, especially considering that the address does not appear to be
invariant, I suspect the driver sets up some I/O from (presumably)
uninitialized data. This goes unnoticed until DMA translation comes
into play. Tamas - did you try enabling DMA translation in Linux
when not running on top of Xen? Depending on the
CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the
default.

> In the meantime, you can try to add to the command line:
> 
> rmrr=0x428f926=0:0:2.0
> 
> In order to force an iommu mapping of this address.

I suspect this won't help much.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 4/4] x86/iommu: add reserved dom0-iommu option to map reserved memory ranges

[Jan Beulich]

>>> On 08.08.18 at 11:53, <roger.pau@citrix.com> wrote:
> I've lost a chunk here during one of the rebases, so the following
> diff should be added to the patch in order to create maps if the iommu
> page tables are shared:
> 
> diff --git a/xen/drivers/passthrough/x86/iommu.c 
> b/xen/drivers/passthrough/x86/iommu.c
> index 6aec43ed1a..6271d8b671 100644
> --- a/xen/drivers/passthrough/x86/iommu.c
> +++ b/xen/drivers/passthrough/x86/iommu.c
> @@ -209,7 +209,13 @@ void __hwdom_init arch_iommu_hwdom_init(struct domain *d)
>          if ( !hwdom_iommu_map(d, pfn, max_pfn) )
>              continue;
>  
> -        rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
> +        if ( iommu_use_hap_pt(d) )
> +        {
> +            ASSERT(is_hvm_domain(d));
> +            rc = set_identity_p2m_entry(d, pfn, p2m_access_rw, 0);
> +        }
> +        else
> +            rc = iommu_map_page(d, pfn, pfn, IOMMUF_readable|IOMMUF_writable);
>          if ( rc )
>              printk(XENLOG_WARNING " d%d: IOMMU mapping failed: %d\n",
>                     d->domain_id, rc);
> 
> I can resend the series in order to ease review.

This would indeed be nice, thanks.

Jan




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich <JBeulich@suse.com> wrote:
>
> >>> On 08.08.18 at 10:25, <roger.pau@citrix.com> wrote:
> > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
> >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
> >> <tamas.k.lengyel@gmail.com> wrote:
> >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> >> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> >> 428f926000, iommu reg = ffff82c000a0c000
> >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> >> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
> >> (XEN)     root_entry[00] = 43aaae001
> >> (XEN)     context[10] = 2_43cf92001
> >> (XEN)     l4[000] = 9c0000043cf91107
> >> (XEN)     l3[10a] = 8000000000000000
> >> (XEN)     l3[10a] not present
> >>
> >> The fault is repeated a million times per second and the system is
> >> pretty much stalled.
> >
> > As Jan says, this page is outside of any range in the memory map. I
> > wonder however what's in there.
> >
> > I think (also seeing the PV issues) you should bring this up with the
> > driver maintainers, it might actually be a bug that the driver is
> > trying to access such address.
>
> Right, especially considering that the address does not appear to be
> invariant, I suspect the driver sets up some I/O from (presumably)
> uninitialized data. This goes unnoticed until DMA translation comes
> into play. Tamas - did you try enabling DMA translation in Linux
> when not running on top of Xen? Depending on the
> CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the
> default.

I checked and this kernel option is not enabled. Are you suggesting to
try booting just Linux with this option enabled to see what happens?

>
> > In the meantime, you can try to add to the command line:
> >
> > rmrr=0x428f926=0:0:2.0
> >
> > In order to force an iommu mapping of this address.
>
> I suspect this won't help much.
>

The mfn is not always the same but seems to be at least on some
occasion. I managed to reboot with the right rmrr= option set but I'm
still getting the same faults over and over for that mfn I set in the
rmrr= option.

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Jan Beulich]

>>> On 15.08.18 at 03:00, <tamas.k.lengyel@gmail.com> wrote:
> On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich <JBeulich@suse.com> wrote:
>>
>> >>> On 08.08.18 at 10:25, <roger.pau@citrix.com> wrote:
>> > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
>> >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
>> >> <tamas.k.lengyel@gmail.com> wrote:
>> >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
>> >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
>> >> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
>> >> 428f926000, iommu reg = ffff82c000a0c000
>> >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
>> >> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
>> >> (XEN)     root_entry[00] = 43aaae001
>> >> (XEN)     context[10] = 2_43cf92001
>> >> (XEN)     l4[000] = 9c0000043cf91107
>> >> (XEN)     l3[10a] = 8000000000000000
>> >> (XEN)     l3[10a] not present
>> >>
>> >> The fault is repeated a million times per second and the system is
>> >> pretty much stalled.
>> >
>> > As Jan says, this page is outside of any range in the memory map. I
>> > wonder however what's in there.
>> >
>> > I think (also seeing the PV issues) you should bring this up with the
>> > driver maintainers, it might actually be a bug that the driver is
>> > trying to access such address.
>>
>> Right, especially considering that the address does not appear to be
>> invariant, I suspect the driver sets up some I/O from (presumably)
>> uninitialized data. This goes unnoticed until DMA translation comes
>> into play. Tamas - did you try enabling DMA translation in Linux
>> when not running on top of Xen? Depending on the
>> CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the
>> default.
> 
> I checked and this kernel option is not enabled. Are you suggesting to
> try booting just Linux with this option enabled to see what happens?

Well, you don't need to rebuild the kernel with the config option
enabled, you can use what you have and add "intel_iommu=on"
to the kernel command line. In case this works without any faults,
changing to "intel_iommu=on,igfx_off" may or may not provide
further insight.

>> > In the meantime, you can try to add to the command line:
>> >
>> > rmrr=0x428f926=0:0:2.0
>> >
>> > In order to force an iommu mapping of this address.
>>
>> I suspect this won't help much.
>>
> 
> The mfn is not always the same but seems to be at least on some
> occasion.

I expect that's because many things early at boot go pretty
deterministically.

> I managed to reboot with the right rmrr= option set but I'm
> still getting the same faults over and over for that mfn I set in the
> rmrr= option.

Hmm, and the faults still show non-present entries for these? This
options is at risk of getting misspelled - did you check that the
option you've specified actually took effect?

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Tamas K Lengyel]

On Wed, Aug 15, 2018 at 12:40 AM Jan Beulich <JBeulich@suse.com> wrote:
>
> >>> On 15.08.18 at 03:00, <tamas.k.lengyel@gmail.com> wrote:
> > On Wed, Aug 8, 2018 at 3:54 AM Jan Beulich <JBeulich@suse.com> wrote:
> >>
> >> >>> On 08.08.18 at 10:25, <roger.pau@citrix.com> wrote:
> >> > On Tue, Aug 07, 2018 at 10:45:32AM -0600, Tamas K Lengyel wrote:
> >> >> On Tue, Aug 7, 2018 at 10:04 AM Tamas K Lengyel
> >> >> <tamas.k.lengyel@gmail.com> wrote:
> >> >> (XEN) [VT-D]iommu.c:919: iommu_fault_status: Fault Overflow
> >> >> (XEN) [VT-D]iommu.c:921: iommu_fault_status: Primary Pending Fault
> >> >> (XEN) [VT-D]DMAR:[DMA Read] Request device [0000:00:02.0] fault addr
> >> >> 428f926000, iommu reg = ffff82c000a0c000
> >> >> (XEN) [VT-D]DMAR: reason 06 - PTE Read access is not set
> >> >> (XEN) print_vtd_entries: iommu #0 dev 0000:00:02.0 gmfn 428f926
> >> >> (XEN)     root_entry[00] = 43aaae001
> >> >> (XEN)     context[10] = 2_43cf92001
> >> >> (XEN)     l4[000] = 9c0000043cf91107
> >> >> (XEN)     l3[10a] = 8000000000000000
> >> >> (XEN)     l3[10a] not present
> >> >>
> >> >> The fault is repeated a million times per second and the system is
> >> >> pretty much stalled.
> >> >
> >> > As Jan says, this page is outside of any range in the memory map. I
> >> > wonder however what's in there.
> >> >
> >> > I think (also seeing the PV issues) you should bring this up with the
> >> > driver maintainers, it might actually be a bug that the driver is
> >> > trying to access such address.
> >>
> >> Right, especially considering that the address does not appear to be
> >> invariant, I suspect the driver sets up some I/O from (presumably)
> >> uninitialized data. This goes unnoticed until DMA translation comes
> >> into play. Tamas - did you try enabling DMA translation in Linux
> >> when not running on top of Xen? Depending on the
> >> CONFIG_INTEL_IOMMU_DEFAULT_ON setting this may not be the
> >> default.
> >
> > I checked and this kernel option is not enabled. Are you suggesting to
> > try booting just Linux with this option enabled to see what happens?
>
> Well, you don't need to rebuild the kernel with the config option
> enabled, you can use what you have and add "intel_iommu=on"
> to the kernel command line. In case this works without any faults,
> changing to "intel_iommu=on,igfx_off" may or may not provide
> further insight.

Booting just linux with both options works just fine, I grepped the
dmesg log for the device that causes the issues with Xen but there are
no errors in the log:

# dmesg | grep "00\:02\.0"
[    0.165669] pci 0000:00:02.0: [8086:5916] type 00 class 0x030000
[    0.165681] pci 0000:00:02.0: reg 0x10: [mem 0xdb000000-0xdbffffff 64bit]
[    0.165687] pci 0000:00:02.0: reg 0x18: [mem 0x90000000-0x9fffffff
64bit pref]
[    0.165692] pci 0000:00:02.0: reg 0x20: [io  0xf000-0xf03f]
[    0.165707] pci 0000:00:02.0: BAR 2: assigned to efifb
[    0.184223] pci 0000:00:02.0: vgaarb: setting as boot VGA device
[    0.184223] pci 0000:00:02.0: vgaarb: VGA device added:
decodes=io+mem,owns=io+mem,locks=none
[    0.184223] pci 0000:00:02.0: vgaarb: bridge control possible
[    0.248925] pci 0000:00:02.0: Video device with shadowed ROM at
[mem 0x000c0000-0x000dffff]
[    1.896026] i915 0000:00:02.0: vgaarb: changed VGA decodes:
olddecodes=io+mem,decodes=io+mem:owns=io+mem
[    1.903881] [drm] Initialized i915 1.6.0 20180514 for 0000:00:02.0 on minor 0
[    3.271519] i915 0000:00:02.0: fb0: inteldrmfb frame buffer device
[   12.519683] snd_hda_intel 0000:00:1f.3: bound 0000:00:02.0 (ops
i915_audio_component_bind_ops [i915])


> >> > In the meantime, you can try to add to the command line:
> >> >
> >> > rmrr=0x428f926=0:0:2.0
> >> >
> >> > In order to force an iommu mapping of this address.
> >>
> >> I suspect this won't help much.
> >>
> >
> > The mfn is not always the same but seems to be at least on some
> > occasion.
>
> I expect that's because many things early at boot go pretty
> deterministically.
>
> > I managed to reboot with the right rmrr= option set but I'm
> > still getting the same faults over and over for that mfn I set in the
> > rmrr= option.
>
> Hmm, and the faults still show non-present entries for these? This
> options is at risk of getting misspelled - did you check that the
> option you've specified actually took effect?

I double checked and the option is set properly but I'm still getting
the same non-present entry faults as before. At the moment I don't
have serial access so not sure how to verify that the option took
effect. This is my boot info:

[pvh]
options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M
dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug
dom0-iommu=relaxed console=vga
rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0
kernel=vmlinuz-4.18.0-rc8 root=/dev/mapper/ubuntu--vg-root ro quiet
splash vt.handoff=1
ramdisk=initrd.img-4.18.0-rc8

Tamas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Jan Beulich]

>>> On 16.08.18 at 18:43, <tamas.k.lengyel@gmail.com> wrote:
> Booting just linux with both options works just fine, I grepped the
> dmesg log for the device that causes the issues with Xen but there are
> no errors in the log:

Well, this as well as ...

> I double checked and the option is set properly but I'm still getting
> the same non-present entry faults as before. At the moment I don't
> have serial access so not sure how to verify that the option took
> effect. This is my boot info:
> 
> [pvh]
> options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M
> dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug
> dom0-iommu=relaxed console=vga
> rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0
> kernel=vmlinuz-4.18.0-rc8 root=/dev/mapper/ubuntu--vg-root ro quiet
> splash vt.handoff=1
> ramdisk=initrd.img-4.18.0-rc8

... this (option looks fine to me) suggest that some debugging is
going to be on order on that system, which I understand will be
at least difficult without serial console.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [PATCH v3 0/4] x86/iommu: PVH Dom0 workarounds for missing RMRR entries

[Roger Pau Monné]

On Thu, Aug 16, 2018 at 10:43:54AM -0600, Tamas K Lengyel wrote:
> I double checked and the option is set properly but I'm still getting
> the same non-present entry faults as before. At the moment I don't
> have serial access so not sure how to verify that the option took
> effect. This is my boot info:
> 
> [pvh]
> options=loglvl=all guest_loglvl=all dom0_mem=4096M,max:4096M
> dom0_max_vcpus=2 sched=null dom0=pvh iommu=required,debug
> dom0-iommu=relaxed console=vga
> rmrr=0x5f48385=0:0:2.0,0x216183=0:0:2.0,0x70b6d19=0:0:2.0

This is not the correct format. It's:

rmrr=0x5f48385=0:0:2.0;0x216183=0:0:2.0;0x70b6d19=0:0:2.0

Note the ';' between rmrr entries.

Roger.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel