[Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools

[Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools

[Peter Xu]

Patch 1: fixes a very rare PT path issue on iova value. It didn't
break anything since it's merely not touched (only if when IOMMU
enabled, then set one device to PT), but still better fix it.

Patch 2-5: added "info iommu" hmp command, and implemented for VT-d.
Meanwhile, added some statistics for iotlb.

Patch 6: introduce "x-iotlb-size" to tune iotlb size, or to turn it
off (e.g., when we want to measure how iotlb affects one payload).

Patch 7: some refine on iotlb entry.

Patch 8: implemented MRU list algorithm for iotlb.

For the last patch, it's logically making more sense than the old
algo, however the performance is merely the same as before (as far as
I tested with simple netperf payloads, in either streaming, rr,
reverse, etc.) since in most normal cases we cannot really let iotlb
overflow especially when size is 1024 by default, e.g., guest kernel
driver will release buffer when after used, and unstrict
intel_iommu=on parameter will also send periodic global iotlb flush
which will reset the whole cache. If anyone has suggestion on specific
workload, please shoot. Anyway, I'm posting this out for review to see
any possible comments/suggestions.

Thanks,

Peter Xu (8):
  intel_iommu: fix VTD_PAGE_MASK
  hmp: add info iommu
  intel_iommu: support "info iommu"
  intel_iommu: add iotlb/context cache statistics
  intel_iommu: hmp: allow "-c" for "info iommu"
  intel_iommu: let iotlb size tunable
  intel_iommu: use access_flags for iotlb
  intel_iommu: implement mru list for iotlb

 hmp-commands-info.hx           |  14 ++++
 hmp.c                          |   6 ++
 hmp.h                          |   1 +
 hw/i386/intel_iommu.c          | 169 +++++++++++++++++++++++++++++++----------
 hw/i386/intel_iommu_internal.h |  11 +--
 hw/i386/trace-events           |   1 -
 hw/i386/x86-iommu.c            |  17 +++++
 include/hw/i386/intel_iommu.h  |  20 ++++-
 include/hw/i386/x86-iommu.h    |   5 ++
 include/hw/iommu.h             |   9 +++
 stubs/Makefile.objs            |   1 +
 stubs/iommu.c                  |   9 +++
 12 files changed, 209 insertions(+), 54 deletions(-)
 create mode 100644 include/hw/iommu.h
 create mode 100644 stubs/iommu.c

-- 
2.7.4

[Qemu-devel] [PATCH 1/8] intel_iommu: fix VTD_PAGE_MASK

[Peter Xu]

IOMMUTLBEntry.iova is returned incorrectly in one PT path (though mostly
we cannot really trigger this path, even if we do, we are mostly
disgarding this value, so it didn't break anything). Fix it by
converting the VTD_PAGE_MASK into normal definition (normally it should
be pfn mask, not offset mask), then switch the other user of it.

Fixes: b93130 ("intel_iommu: cleanup vtd_{do_}iommu_translate()")
Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c          | 2 +-
 hw/i386/intel_iommu_internal.h | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index a9b59bd..a5c83dd 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -1141,7 +1141,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
     if (vtd_ce_get_type(&ce) == VTD_CONTEXT_TT_PASS_THROUGH) {
         entry->iova = addr & VTD_PAGE_MASK;
         entry->translated_addr = entry->iova;
-        entry->addr_mask = VTD_PAGE_MASK;
+        entry->addr_mask = ~VTD_PAGE_MASK;
         entry->perm = IOMMU_RW;
         trace_vtd_translate_pt(source_id, entry->iova);
 
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index f50ecd8..d1d6290 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -384,7 +384,7 @@ typedef struct VTDIOTLBPageInvInfo VTDIOTLBPageInvInfo;
 /* Pagesize of VTD paging structures, including root and context tables */
 #define VTD_PAGE_SHIFT              12
 #define VTD_PAGE_SIZE               (1ULL << VTD_PAGE_SHIFT)
-#define VTD_PAGE_MASK               (VTD_PAGE_SIZE - 1)
+#define VTD_PAGE_MASK               ~(VTD_PAGE_SIZE - 1)
 
 #define VTD_PAGE_SHIFT_4K           12
 #define VTD_PAGE_MASK_4K            (~((1ULL << VTD_PAGE_SHIFT_4K) - 1))
-- 
2.7.4

[Qemu-devel] [PATCH 2/8] hmp: add info iommu

[Peter Xu]

Introducing a new HMP interface "info iommu" to dump IOMMU information.
This command will be only used for developers' debugging purpose, and no
possible use for users. So QMP interface will not be implemented.

This patch only implements the stub one.  We can provide arch-dependent
status dump in the future.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hmp-commands-info.hx | 14 ++++++++++++++
 hmp.c                |  6 ++++++
 hmp.h                |  1 +
 include/hw/iommu.h   |  9 +++++++++
 stubs/Makefile.objs  |  1 +
 stubs/iommu.c        |  9 +++++++++
 6 files changed, 40 insertions(+)
 create mode 100644 include/hw/iommu.h
 create mode 100644 stubs/iommu.c

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index ae16901..a39243d 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -802,6 +802,20 @@ Dump all the ramblocks of the system.
 ETEXI
 
     {
+        .name       = "iommu",
+        .args_type  = "",
+        .params     = "",
+        .help       = "Display system IOMMU information",
+        .cmd        = hmp_info_iommu,
+    },
+
+STEXI
+@item info ramblock
+@findex ramblock
+Dump all the ramblocks of the system.
+ETEXI
+
+    {
         .name       = "hotpluggable-cpus",
         .args_type  = "",
         .params     = "",
diff --git a/hmp.c b/hmp.c
index 8c72c58..68994af 100644
--- a/hmp.c
+++ b/hmp.c
@@ -42,6 +42,7 @@
 #include "qemu/error-report.h"
 #include "exec/ramlist.h"
 #include "hw/intc/intc.h"
+#include "hw/iommu.h"
 #include "migration/snapshot.h"
 
 #ifdef CONFIG_SPICE
@@ -2817,3 +2818,8 @@ void hmp_info_vm_generation_id(Monitor *mon, const QDict *qdict)
     hmp_handle_error(mon, &err);
     qapi_free_GuidInfo(info);
 }
+
+void hmp_info_iommu(Monitor *mon, const QDict *qdict)
+{
+    arch_iommu_info(mon, qdict);
+}
diff --git a/hmp.h b/hmp.h
index d8b94ce..ed01c49 100644
--- a/hmp.h
+++ b/hmp.h
@@ -143,5 +143,6 @@ void hmp_info_dump(Monitor *mon, const QDict *qdict);
 void hmp_info_ramblock(Monitor *mon, const QDict *qdict);
 void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict);
 void hmp_info_vm_generation_id(Monitor *mon, const QDict *qdict);
+void hmp_info_iommu(Monitor *mon, const QDict *qdict);
 
 #endif
diff --git a/include/hw/iommu.h b/include/hw/iommu.h
new file mode 100644
index 0000000..5201a8d
--- /dev/null
+++ b/include/hw/iommu.h
@@ -0,0 +1,9 @@
+#ifndef __HW_IOMMU_H__
+#define __HW_IOMMU_H__
+
+#include "qemu/typedefs.h"
+#include "qapi/qmp/qdict.h"
+
+void arch_iommu_info(Monitor *mon, const QDict *qdict);
+
+#endif
diff --git a/stubs/Makefile.objs b/stubs/Makefile.objs
index f5b47bf..dfd5569 100644
--- a/stubs/Makefile.objs
+++ b/stubs/Makefile.objs
@@ -39,3 +39,4 @@ stub-obj-y += pc_madt_cpu_entry.o
 stub-obj-y += vmgenid.o
 stub-obj-y += xen-common.o
 stub-obj-y += xen-hvm.o
+stub-obj-y += iommu.o
diff --git a/stubs/iommu.c b/stubs/iommu.c
new file mode 100644
index 0000000..75b4f4c
--- /dev/null
+++ b/stubs/iommu.c
@@ -0,0 +1,9 @@
+#include "qemu/osdep.h"
+#include "monitor/monitor.h"
+#include "hw/iommu.h"
+
+void arch_iommu_info(Monitor *mon, const QDict *qdict)
+{
+    monitor_printf(mon, "This command is not supported "
+                   "on this platform.\n");
+}
-- 
2.7.4

[Qemu-devel] [PATCH 3/8] intel_iommu: support "info iommu"

[Peter Xu]

Dump critical information for VT-d. Sample output:

(qemu) info iommu
Version: 1
Cap: 0x12008c22260286
Extended Cap: 0xf00f5a
DMAR: enabled, root=0x7435f000 (extended=0)
IR: enabled, root=0x17a400000, size=0x10000 (eim=1)
QI: enabled, root=0x17aadf000, head=156, tail=156, size=256
Caching-mode: enabled
Misc: next_frr=0, context_gen=2, buggy_eim=0

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c       | 39 +++++++++++++++++++++++++++++++++++++++
 hw/i386/x86-iommu.c         | 17 +++++++++++++++++
 include/hw/i386/x86-iommu.h |  5 +++++
 3 files changed, 61 insertions(+)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index a5c83dd..39f772a 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -2991,6 +2991,44 @@ static bool vtd_decide_config(IntelIOMMUState *s, Error **errp)
     return true;
 }
 
+#define DUMP(...) monitor_printf(mon, ## __VA_ARGS__)
+static void vtd_info_dump(X86IOMMUState *x86_iommu, Monitor *mon,
+                          const QDict *qdict)
+{
+    IntelIOMMUState *s = INTEL_IOMMU_DEVICE(x86_iommu);
+
+    DUMP("Version: %d\n", 1);
+    DUMP("Cap: 0x%"PRIx64"\n", s->cap);
+    DUMP("Extended Cap: 0x%"PRIx64"\n", s->ecap);
+
+    DUMP("DMAR: %s", s->dmar_enabled ? "enabled" : "disabled");
+    if (s->dmar_enabled) {
+        DUMP(", root=0x%"PRIx64" (extended=%d)",
+             s->root, s->root_extended);
+    }
+    DUMP("\n");
+
+    DUMP("IR: %s", s->intr_enabled ? "enabled" : "disabled");
+    if (s->intr_enabled) {
+        DUMP(", root=0x%"PRIx64", size=0x%"PRIx32" (eim=%d)",
+             s->intr_root, s->intr_size, s->intr_eime);
+    }
+    DUMP("\n");
+
+    DUMP("QI: %s", s->qi_enabled ? "enabled" : "disabled");
+    if (s->qi_enabled) {
+        DUMP(", root=0x%"PRIx64", head=%u, tail=%u, size=%u",
+             s->iq, s->iq_head, s->iq_tail, s->iq_size);
+    }
+    DUMP("\n");
+
+    DUMP("Caching-mode: %s\n", s->caching_mode ? "enabled" : "disabled");
+    DUMP("Misc: next_frr=%d, context_gen=%d, buggy_eim=%d\n",
+         s->next_frcd_reg, s->context_cache_gen, s->buggy_eim);
+    DUMP("      iotlb_size=%d\n", g_hash_table_size(s->iotlb));
+}
+#undef DUMP
+
 static void vtd_realize(DeviceState *dev, Error **errp)
 {
     MachineState *ms = MACHINE(qdev_get_machine());
@@ -3042,6 +3080,7 @@ static void vtd_class_init(ObjectClass *klass, void *data)
     dc->hotpluggable = false;
     x86_class->realize = vtd_realize;
     x86_class->int_remap = vtd_int_remap;
+    x86_class->info_dump = vtd_info_dump;
     /* Supported by the pc-q35-* machine types */
     dc->user_creatable = true;
 }
diff --git a/hw/i386/x86-iommu.c b/hw/i386/x86-iommu.c
index 293caf8..fed35b4 100644
--- a/hw/i386/x86-iommu.c
+++ b/hw/i386/x86-iommu.c
@@ -76,6 +76,23 @@ IommuType x86_iommu_get_type(void)
     return x86_iommu_default->type;
 }
 
+void arch_iommu_info(Monitor *mon, const QDict *qdict)
+{
+    X86IOMMUState *iommu = x86_iommu_get_default();
+    X86IOMMUClass *class;
+
+    if (!iommu) {
+        monitor_printf(mon, "No IOMMU is detected.\n");
+        return;
+    }
+
+    class = X86_IOMMU_GET_CLASS(iommu);
+
+    if (class->info_dump) {
+        class->info_dump(iommu, mon, qdict);
+    }
+}
+
 static void x86_iommu_realize(DeviceState *dev, Error **errp)
 {
     X86IOMMUState *x86_iommu = X86_IOMMU_DEVICE(dev);
diff --git a/include/hw/i386/x86-iommu.h b/include/hw/i386/x86-iommu.h
index ef89c0c..c414b65 100644
--- a/include/hw/i386/x86-iommu.h
+++ b/include/hw/i386/x86-iommu.h
@@ -22,6 +22,8 @@
 
 #include "hw/sysbus.h"
 #include "hw/pci/pci.h"
+#include "hw/iommu.h"
+#include "monitor/monitor.h"
 
 #define  TYPE_X86_IOMMU_DEVICE  ("x86-iommu")
 #define  X86_IOMMU_DEVICE(obj) \
@@ -50,6 +52,9 @@ struct X86IOMMUClass {
     /* MSI-based interrupt remapping */
     int (*int_remap)(X86IOMMUState *iommu, MSIMessage *src,
                      MSIMessage *dst, uint16_t sid);
+    /* Dump IOMMU information */
+    void (*info_dump)(X86IOMMUState *iommu, Monitor *mon,
+                      const QDict *qdict);
 };
 
 /**
-- 
2.7.4

[Qemu-devel] [PATCH 4/8] intel_iommu: add iotlb/context cache statistics

[Peter Xu]

Add statistics for the VT-d IOMMU DMA remapping.

Now "info iommu" shows us this for extra:

Statistics: iotlb=26.35% (6689/25388), context=99.99% (18697/18699)

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c         | 21 +++++++++++++++++++++
 include/hw/i386/intel_iommu.h | 10 ++++++++++
 2 files changed, 31 insertions(+)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 39f772a..45d0919 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -37,6 +37,11 @@
 #include "kvm_i386.h"
 #include "trace.h"
 
+static void vtd_reset_stats(IntelIOMMUState *s)
+{
+    memset(&s->cache_stat, 0, sizeof(s->cache_stat));
+}
+
 static void vtd_define_quad(IntelIOMMUState *s, hwaddr addr, uint64_t val,
                             uint64_t wmask, uint64_t w1cmask)
 {
@@ -1095,9 +1100,12 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
      */
     assert(!vtd_is_interrupt_addr(addr));
 
+    s->cache_stat.iotlb_total++;
+
     /* Try to fetch slpte form IOTLB */
     iotlb_entry = vtd_lookup_iotlb(s, source_id, addr);
     if (iotlb_entry) {
+        s->cache_stat.iotlb_hit++;
         trace_vtd_iotlb_page_hit(source_id, addr, iotlb_entry->slpte,
                                  iotlb_entry->domain_id);
         slpte = iotlb_entry->slpte;
@@ -1107,8 +1115,11 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         goto out;
     }
 
+    s->cache_stat.context_total++;
+
     /* Try to fetch context-entry from cache first */
     if (cc_entry->context_cache_gen == s->context_cache_gen) {
+        s->cache_stat.context_hit++;
         trace_vtd_iotlb_cc_hit(bus_num, devfn, cc_entry->context_entry.hi,
                                cc_entry->context_entry.lo,
                                cc_entry->context_cache_gen);
@@ -2875,6 +2886,7 @@ static void vtd_init(IntelIOMMUState *s)
 
     vtd_reset_context_cache(s);
     vtd_reset_iotlb(s);
+    vtd_reset_stats(s);
 
     /* Define registers with default values and bit semantics */
     vtd_define_long(s, DMAR_VER_REG, 0x10UL, 0, 0);
@@ -3022,6 +3034,15 @@ static void vtd_info_dump(X86IOMMUState *x86_iommu, Monitor *mon,
     }
     DUMP("\n");
 
+    DUMP("Statistics: iotlb=%.2lf%% (%"PRIu64"/%"PRIu64"), "
+         "context=%.2lf%% (%"PRIu64"/%"PRIu64")\n",
+         (double)s->cache_stat.iotlb_hit /
+         s->cache_stat.iotlb_total * 100,
+         s->cache_stat.iotlb_hit, s->cache_stat.iotlb_total,
+         (double)s->cache_stat.context_hit /
+         s->cache_stat.context_total * 100,
+         s->cache_stat.context_hit, s->cache_stat.context_total);
+
     DUMP("Caching-mode: %s\n", s->caching_mode ? "enabled" : "disabled");
     DUMP("Misc: next_frr=%d, context_gen=%d, buggy_eim=%d\n",
          s->next_frcd_reg, s->context_cache_gen, s->buggy_eim);
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 3e51876..fc69ff3 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -255,6 +255,13 @@ struct IntelIOMMUNotifierNode {
     QLIST_ENTRY(IntelIOMMUNotifierNode) next;
 };
 
+typedef struct IOMMUCacheStats {
+    uint64_t iotlb_hit;
+    uint64_t iotlb_total;
+    uint64_t context_hit;
+    uint64_t context_total;
+} IOMMUCacheStats;
+
 /* The iommu (DMAR) device state struct */
 struct IntelIOMMUState {
     X86IOMMUState x86_iommu;
@@ -302,6 +309,9 @@ struct IntelIOMMUState {
     bool intr_eime;                 /* Extended interrupt mode enabled */
     OnOffAuto intr_eim;             /* Toggle for EIM cabability */
     bool buggy_eim;                 /* Force buggy EIM unless eim=off */
+
+    /* For statistics */
+    IOMMUCacheStats cache_stat;
 };
 
 /* Find the VTD Address space associated with the given bus pointer,
-- 
2.7.4

[Qemu-devel] [PATCH 5/8] intel_iommu: hmp: allow "-c" for "info iommu"

[Peter Xu]

New parameter "-c" for it to clear statistics. Other platforms can
selectively support this (though none yet).

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hmp-commands-info.hx  | 4 ++--
 hw/i386/intel_iommu.c | 5 +++++
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
index a39243d..2add941 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -803,8 +803,8 @@ ETEXI
 
     {
         .name       = "iommu",
-        .args_type  = "",
-        .params     = "",
+        .args_type  = "clear_stats:-c",
+        .params     = "[-c]",
         .help       = "Display system IOMMU information",
         .cmd        = hmp_info_iommu,
     },
diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 45d0919..72b39f0 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -3008,6 +3008,7 @@ static void vtd_info_dump(X86IOMMUState *x86_iommu, Monitor *mon,
                           const QDict *qdict)
 {
     IntelIOMMUState *s = INTEL_IOMMU_DEVICE(x86_iommu);
+    bool clear_stats = qdict_get_try_bool(qdict, "clear_stats", false);
 
     DUMP("Version: %d\n", 1);
     DUMP("Cap: 0x%"PRIx64"\n", s->cap);
@@ -3047,6 +3048,10 @@ static void vtd_info_dump(X86IOMMUState *x86_iommu, Monitor *mon,
     DUMP("Misc: next_frr=%d, context_gen=%d, buggy_eim=%d\n",
          s->next_frcd_reg, s->context_cache_gen, s->buggy_eim);
     DUMP("      iotlb_size=%d\n", g_hash_table_size(s->iotlb));
+
+    if (clear_stats) {
+        vtd_reset_stats(s);
+    }
 }
 #undef DUMP
 
-- 
2.7.4

[Qemu-devel] [PATCH 6/8] intel_iommu: let iotlb size tunable

[Peter Xu]

We were having static IOTLB size as 1024. Let it be a tunable. We can
also turns IOTLB off if we want, by specify the size as zero.

The tunable is named as "x-iotlb-size" since that should not really be
something used by user yet, but mostly for debugging purpose now.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c          | 14 ++++++++++++--
 hw/i386/intel_iommu_internal.h |  1 -
 include/hw/i386/intel_iommu.h  |  1 +
 3 files changed, 13 insertions(+), 3 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index 72b39f0..fc05764 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -227,6 +227,10 @@ static VTDIOTLBEntry *vtd_lookup_iotlb(IntelIOMMUState *s, uint16_t source_id,
     uint64_t key;
     int level;
 
+    if (s->iotlb_size == 0) {
+        return NULL;
+    }
+
     for (level = VTD_SL_PT_LEVEL; level < VTD_SL_PML4_LEVEL; level++) {
         key = vtd_get_iotlb_key(vtd_get_iotlb_gfn(addr, level),
                                 source_id, level);
@@ -249,8 +253,12 @@ static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
     uint64_t *key = g_malloc(sizeof(*key));
     uint64_t gfn = vtd_get_iotlb_gfn(addr, level);
 
+    if (s->iotlb_size == 0) {
+        return;
+    }
+
     trace_vtd_iotlb_page_update(source_id, addr, slpte, domain_id);
-    if (g_hash_table_size(s->iotlb) >= VTD_IOTLB_MAX_SIZE) {
+    if (g_hash_table_size(s->iotlb) >= s->iotlb_size) {
         trace_vtd_iotlb_reset("iotlb exceeds size limit");
         vtd_reset_iotlb(s);
     }
@@ -2388,6 +2396,7 @@ static Property vtd_properties[] = {
                             ON_OFF_AUTO_AUTO),
     DEFINE_PROP_BOOL("x-buggy-eim", IntelIOMMUState, buggy_eim, false),
     DEFINE_PROP_BOOL("caching-mode", IntelIOMMUState, caching_mode, FALSE),
+    DEFINE_PROP_UINT16("x-iotlb-size", IntelIOMMUState, iotlb_size, 1024),
     DEFINE_PROP_END_OF_LIST(),
 };
 
@@ -3047,7 +3056,8 @@ static void vtd_info_dump(X86IOMMUState *x86_iommu, Monitor *mon,
     DUMP("Caching-mode: %s\n", s->caching_mode ? "enabled" : "disabled");
     DUMP("Misc: next_frr=%d, context_gen=%d, buggy_eim=%d\n",
          s->next_frcd_reg, s->context_cache_gen, s->buggy_eim);
-    DUMP("      iotlb_size=%d\n", g_hash_table_size(s->iotlb));
+    DUMP("      iotlb_size=%d/%d\n", g_hash_table_size(s->iotlb),
+         s->iotlb_size);
 
     if (clear_stats) {
         vtd_reset_stats(s);
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index d1d6290..dc0257c 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -116,7 +116,6 @@
 /* The shift of source_id in the key of IOTLB hash table */
 #define VTD_IOTLB_SID_SHIFT         36
 #define VTD_IOTLB_LVL_SHIFT         52
-#define VTD_IOTLB_MAX_SIZE          1024    /* Max size of the hash table */
 
 /* IOTLB_REG */
 #define VTD_TLB_GLOBAL_FLUSH        (1ULL << 60) /* Global invalidation */
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index fc69ff3..947c153 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -295,6 +295,7 @@ struct IntelIOMMUState {
 
     uint32_t context_cache_gen;     /* Should be in [1,MAX] */
     GHashTable *iotlb;              /* IOTLB */
+    uint16_t iotlb_size;            /* IOTLB max cache entries */
 
     MemoryRegionIOMMUOps iommu_ops;
     GHashTable *vtd_as_by_busptr;   /* VTDBus objects indexed by PCIBus* reference */
-- 
2.7.4

[Qemu-devel] [PATCH 7/8] intel_iommu: use access_flags for iotlb

[Peter Xu]

It was cached by read/write separately. Let's merge them.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c         | 15 +++++++--------
 include/hw/i386/intel_iommu.h |  3 +--
 2 files changed, 8 insertions(+), 10 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index fc05764..c2b2683 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -246,8 +246,7 @@ out:
 
 static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
                              uint16_t domain_id, hwaddr addr, uint64_t slpte,
-                             bool read_flags, bool write_flags,
-                             uint32_t level)
+                             uint8_t access_flags, uint32_t level)
 {
     VTDIOTLBEntry *entry = g_malloc(sizeof(*entry));
     uint64_t *key = g_malloc(sizeof(*key));
@@ -266,8 +265,7 @@ static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
     entry->gfn = gfn;
     entry->domain_id = domain_id;
     entry->slpte = slpte;
-    entry->read_flags = read_flags;
-    entry->write_flags = write_flags;
+    entry->access_flags = access_flags;
     entry->mask = vtd_slpt_level_page_mask(level);
     *key = vtd_get_iotlb_key(gfn, source_id, level);
     g_hash_table_replace(s->iotlb, key, entry);
@@ -1100,6 +1098,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
     bool is_fpd_set = false;
     bool reads = true;
     bool writes = true;
+    uint8_t access_flags;
     VTDIOTLBEntry *iotlb_entry;
 
     /*
@@ -1117,8 +1116,7 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
         trace_vtd_iotlb_page_hit(source_id, addr, iotlb_entry->slpte,
                                  iotlb_entry->domain_id);
         slpte = iotlb_entry->slpte;
-        reads = iotlb_entry->read_flags;
-        writes = iotlb_entry->write_flags;
+        access_flags = iotlb_entry->access_flags;
         page_mask = iotlb_entry->mask;
         goto out;
     }
@@ -1191,13 +1189,14 @@ static bool vtd_do_iommu_translate(VTDAddressSpace *vtd_as, PCIBus *bus,
     }
 
     page_mask = vtd_slpt_level_page_mask(level);
+    access_flags = IOMMU_ACCESS_FLAG(reads, writes);
     vtd_update_iotlb(s, source_id, VTD_CONTEXT_ENTRY_DID(ce.hi), addr, slpte,
-                     reads, writes, level);
+                     access_flags, level);
 out:
     entry->iova = addr & page_mask;
     entry->translated_addr = vtd_get_slpte_addr(slpte) & page_mask;
     entry->addr_mask = ~page_mask;
-    entry->perm = IOMMU_ACCESS_FLAG(reads, writes);
+    entry->perm = access_flags;
     return true;
 
 error:
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 947c153..4960f8d 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -101,8 +101,7 @@ struct VTDIOTLBEntry {
     uint16_t domain_id;
     uint64_t slpte;
     uint64_t mask;
-    bool read_flags;
-    bool write_flags;
+    uint8_t access_flags;
 };
 
 /* VT-d Source-ID Qualifier types */
-- 
2.7.4

[Qemu-devel] [PATCH 8/8] intel_iommu: implement mru list for iotlb

[Peter Xu]

It is not wise to disgard all the IOTLB cache when cache size reaches
max, but that's what we do now. A slightly better (but still simple) way
to do this is, we just throw away the least recent used cache entry.

This patch implemented MRU list algorithm for VT-d IOTLB. The main logic
is to maintain a Most Recently Used list for the IOTLB entries. The hash
table is still used for lookup, though a new list field is added to each
IOTLB entry for a iotlb MRU list. For each active IOTLB entry, it's both
in the hash table in s->iotlb, and also linked into the MRU list of
s->iotlb_head. The hash helps in fast lookup, and the MRU list helps in
knowing whether the cache is still hot.

After we have such a MRU list, replacing all the iterators of IOTLB
entries by using list iterations rather than hash table iterations.

Signed-off-by: Peter Xu <peterx@redhat.com>
---
 hw/i386/intel_iommu.c          | 75 +++++++++++++++++++++++++-----------------
 hw/i386/intel_iommu_internal.h |  8 -----
 hw/i386/trace-events           |  1 -
 include/hw/i386/intel_iommu.h  |  6 +++-
 4 files changed, 50 insertions(+), 40 deletions(-)

diff --git a/hw/i386/intel_iommu.c b/hw/i386/intel_iommu.c
index c2b2683..8375fc3 100644
--- a/hw/i386/intel_iommu.c
+++ b/hw/i386/intel_iommu.c
@@ -37,6 +37,9 @@
 #include "kvm_i386.h"
 #include "trace.h"
 
+#define FOREACH_IOTLB_SAFE(entry, s, entry_n) \
+    QTAILQ_FOREACH_SAFE(entry, &(s)->iotlb_head, link, entry_n)
+
 static void vtd_reset_stats(IntelIOMMUState *s)
 {
     memset(&s->cache_stat, 0, sizeof(s->cache_stat));
@@ -144,14 +147,6 @@ static guint vtd_uint64_hash(gconstpointer v)
     return (guint)*(const uint64_t *)v;
 }
 
-static gboolean vtd_hash_remove_by_domain(gpointer key, gpointer value,
-                                          gpointer user_data)
-{
-    VTDIOTLBEntry *entry = (VTDIOTLBEntry *)value;
-    uint16_t domain_id = *(uint16_t *)user_data;
-    return entry->domain_id == domain_id;
-}
-
 /* The shift of an addr for a certain level of paging structure */
 static inline uint32_t vtd_slpt_level_shift(uint32_t level)
 {
@@ -164,18 +159,6 @@ static inline uint64_t vtd_slpt_level_page_mask(uint32_t level)
     return ~((1ULL << vtd_slpt_level_shift(level)) - 1);
 }
 
-static gboolean vtd_hash_remove_by_page(gpointer key, gpointer value,
-                                        gpointer user_data)
-{
-    VTDIOTLBEntry *entry = (VTDIOTLBEntry *)value;
-    VTDIOTLBPageInvInfo *info = (VTDIOTLBPageInvInfo *)user_data;
-    uint64_t gfn = (info->addr >> VTD_PAGE_SHIFT_4K) & info->mask;
-    uint64_t gfn_tlb = (info->addr & entry->mask) >> VTD_PAGE_SHIFT_4K;
-    return (entry->domain_id == info->domain_id) &&
-            (((entry->gfn & info->mask) == gfn) ||
-             (entry->gfn == gfn_tlb));
-}
-
 /* Reset all the gen of VTDAddressSpace to zero and set the gen of
  * IntelIOMMUState to 1.
  */
@@ -206,6 +189,7 @@ static void vtd_reset_iotlb(IntelIOMMUState *s)
 {
     assert(s->iotlb);
     g_hash_table_remove_all(s->iotlb);
+    QTAILQ_INIT(&s->iotlb_head);
 }
 
 static uint64_t vtd_get_iotlb_key(uint64_t gfn, uint16_t source_id,
@@ -236,6 +220,9 @@ static VTDIOTLBEntry *vtd_lookup_iotlb(IntelIOMMUState *s, uint16_t source_id,
                                 source_id, level);
         entry = g_hash_table_lookup(s->iotlb, &key);
         if (entry) {
+            /* Move the entry to the head of MRU list */
+            QTAILQ_REMOVE(&s->iotlb_head, entry, link);
+            QTAILQ_INSERT_HEAD(&s->iotlb_head, entry, link);
             goto out;
         }
     }
@@ -244,11 +231,23 @@ out:
     return entry;
 }
 
+static void vtd_iotlb_remove_entry(IntelIOMMUState *s, VTDIOTLBEntry *entry)
+{
+    uint64_t key = entry->key;
+
+    /*
+     * To remove an entry, we need to both remove it from the MRU
+     * list, and also from the hash table.
+     */
+    QTAILQ_REMOVE(&s->iotlb_head, entry, link);
+    g_hash_table_remove(s->iotlb, &key);
+}
+
 static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
                              uint16_t domain_id, hwaddr addr, uint64_t slpte,
                              uint8_t access_flags, uint32_t level)
 {
-    VTDIOTLBEntry *entry = g_malloc(sizeof(*entry));
+    VTDIOTLBEntry *entry = g_new0(VTDIOTLBEntry, 1), *last;
     uint64_t *key = g_malloc(sizeof(*key));
     uint64_t gfn = vtd_get_iotlb_gfn(addr, level);
 
@@ -258,8 +257,9 @@ static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
 
     trace_vtd_iotlb_page_update(source_id, addr, slpte, domain_id);
     if (g_hash_table_size(s->iotlb) >= s->iotlb_size) {
-        trace_vtd_iotlb_reset("iotlb exceeds size limit");
-        vtd_reset_iotlb(s);
+        /* Remove the Least Recently Used cache */
+        last = QTAILQ_LAST(&s->iotlb_head, VTDIOTLBEntryHead);
+        vtd_iotlb_remove_entry(s, last);
     }
 
     entry->gfn = gfn;
@@ -268,7 +268,11 @@ static void vtd_update_iotlb(IntelIOMMUState *s, uint16_t source_id,
     entry->access_flags = access_flags;
     entry->mask = vtd_slpt_level_page_mask(level);
     *key = vtd_get_iotlb_key(gfn, source_id, level);
+    entry->key = *key;
     g_hash_table_replace(s->iotlb, key, entry);
+
+    /* Update MRU list */
+    QTAILQ_INSERT_HEAD(&s->iotlb_head, entry, link);
 }
 
 /* Given the reg addr of both the message data and address, generate an
@@ -1365,11 +1369,15 @@ static void vtd_iotlb_domain_invalidate(IntelIOMMUState *s, uint16_t domain_id)
     IntelIOMMUNotifierNode *node;
     VTDContextEntry ce;
     VTDAddressSpace *vtd_as;
+    VTDIOTLBEntry *entry, *entry_n;
 
     trace_vtd_inv_desc_iotlb_domain(domain_id);
 
-    g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_domain,
-                                &domain_id);
+    FOREACH_IOTLB_SAFE(entry, s, entry_n) {
+        if (entry->domain_id == domain_id) {
+            vtd_iotlb_remove_entry(s, entry);
+        }
+    }
 
     QLIST_FOREACH(node, &s->notifiers_list, next) {
         vtd_as = node->vtd_as;
@@ -1411,15 +1419,22 @@ static void vtd_iotlb_page_invalidate_notify(IntelIOMMUState *s,
 static void vtd_iotlb_page_invalidate(IntelIOMMUState *s, uint16_t domain_id,
                                       hwaddr addr, uint8_t am)
 {
-    VTDIOTLBPageInvInfo info;
+    VTDIOTLBEntry *entry, *entry_n;
+    uint64_t gfn, mask;
 
     trace_vtd_inv_desc_iotlb_pages(domain_id, addr, am);
 
     assert(am <= VTD_MAMV);
-    info.domain_id = domain_id;
-    info.addr = addr;
-    info.mask = ~((1 << am) - 1);
-    g_hash_table_foreach_remove(s->iotlb, vtd_hash_remove_by_page, &info);
+
+    mask = ~((1 << am) - 1);
+    gfn = (addr >> VTD_PAGE_SHIFT) & mask;
+
+    FOREACH_IOTLB_SAFE(entry, s, entry_n) {
+        if (entry->domain_id == domain_id && (entry->gfn & mask) == gfn) {
+            vtd_iotlb_remove_entry(s, entry);
+        }
+    }
+
     vtd_iotlb_page_invalidate_notify(s, domain_id, addr, am);
 }
 
diff --git a/hw/i386/intel_iommu_internal.h b/hw/i386/intel_iommu_internal.h
index dc0257c..84e68e1 100644
--- a/hw/i386/intel_iommu_internal.h
+++ b/hw/i386/intel_iommu_internal.h
@@ -372,14 +372,6 @@ typedef union VTDInvDesc VTDInvDesc;
 #define VTD_INV_DESC_DEVICE_IOTLB_RSVD_HI 0xffeULL
 #define VTD_INV_DESC_DEVICE_IOTLB_RSVD_LO 0xffff0000ffe0fff8
 
-/* Information about page-selective IOTLB invalidate */
-struct VTDIOTLBPageInvInfo {
-    uint16_t domain_id;
-    uint64_t addr;
-    uint8_t mask;
-};
-typedef struct VTDIOTLBPageInvInfo VTDIOTLBPageInvInfo;
-
 /* Pagesize of VTD paging structures, including root and context tables */
 #define VTD_PAGE_SHIFT              12
 #define VTD_PAGE_SIZE               (1ULL << VTD_PAGE_SHIFT)
diff --git a/hw/i386/trace-events b/hw/i386/trace-events
index 5f111d6..edf0dca 100644
--- a/hw/i386/trace-events
+++ b/hw/i386/trace-events
@@ -34,7 +34,6 @@ vtd_iotlb_page_hit(uint16_t sid, uint64_t addr, uint64_t slpte, uint16_t domain)
 vtd_iotlb_page_update(uint16_t sid, uint64_t addr, uint64_t slpte, uint16_t domain) "IOTLB page update sid 0x%"PRIx16" iova 0x%"PRIx64" slpte 0x%"PRIx64" domain 0x%"PRIx16
 vtd_iotlb_cc_hit(uint8_t bus, uint8_t devfn, uint64_t high, uint64_t low, uint32_t gen) "IOTLB context hit bus 0x%"PRIx8" devfn 0x%"PRIx8" high 0x%"PRIx64" low 0x%"PRIx64" gen %"PRIu32
 vtd_iotlb_cc_update(uint8_t bus, uint8_t devfn, uint64_t high, uint64_t low, uint32_t gen1, uint32_t gen2) "IOTLB context update bus 0x%"PRIx8" devfn 0x%"PRIx8" high 0x%"PRIx64" low 0x%"PRIx64" gen %"PRIu32" -> gen %"PRIu32
-vtd_iotlb_reset(const char *reason) "IOTLB reset (reason: %s)"
 vtd_fault_disabled(void) "Fault processing disabled for context entry"
 vtd_replay_ce_valid(uint8_t bus, uint8_t dev, uint8_t fn, uint16_t domain, uint64_t hi, uint64_t lo) "replay valid context device %02"PRIx8":%02"PRIx8".%02"PRIx8" domain 0x%"PRIx16" hi 0x%"PRIx64" lo 0x%"PRIx64
 vtd_replay_ce_invalid(uint8_t bus, uint8_t dev, uint8_t fn) "replay invalid context device %02"PRIx8":%02"PRIx8".%02"PRIx8
diff --git a/include/hw/i386/intel_iommu.h b/include/hw/i386/intel_iommu.h
index 4960f8d..bd17a24 100644
--- a/include/hw/i386/intel_iommu.h
+++ b/include/hw/i386/intel_iommu.h
@@ -98,9 +98,11 @@ struct VTDBus {
 
 struct VTDIOTLBEntry {
     uint64_t gfn;
-    uint16_t domain_id;
     uint64_t slpte;
     uint64_t mask;
+    uint64_t key;
+    QTAILQ_ENTRY(VTDIOTLBEntry) link;
+    uint16_t domain_id;
     uint8_t access_flags;
 };
 
@@ -295,6 +297,8 @@ struct IntelIOMMUState {
     uint32_t context_cache_gen;     /* Should be in [1,MAX] */
     GHashTable *iotlb;              /* IOTLB */
     uint16_t iotlb_size;            /* IOTLB max cache entries */
+    /* Head of IOTLB MRU list */
+    QTAILQ_HEAD(VTDIOTLBEntryHead, VTDIOTLBEntry) iotlb_head;
 
     MemoryRegionIOMMUOps iommu_ops;
     GHashTable *vtd_as_by_busptr;   /* VTDBus objects indexed by PCIBus* reference */
-- 
2.7.4

Re: [Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools

[Peter Xu]

On Tue, Jun 27, 2017 at 05:03:31PM +0800, Peter Xu wrote:
> Patch 1: fixes a very rare PT path issue on iova value. It didn't
> break anything since it's merely not touched (only if when IOMMU
> enabled, then set one device to PT), but still better fix it.
> 
> Patch 2-5: added "info iommu" hmp command, and implemented for VT-d.
> Meanwhile, added some statistics for iotlb.

For patch 2,3,5 I should CC Dave. Sorry to have forgotten.

> 
> Patch 6: introduce "x-iotlb-size" to tune iotlb size, or to turn it
> off (e.g., when we want to measure how iotlb affects one payload).
> 
> Patch 7: some refine on iotlb entry.
> 
> Patch 8: implemented MRU list algorithm for iotlb.
> 
> For the last patch, it's logically making more sense than the old
> algo, however the performance is merely the same as before (as far as
> I tested with simple netperf payloads, in either streaming, rr,
> reverse, etc.) since in most normal cases we cannot really let iotlb
> overflow especially when size is 1024 by default, e.g., guest kernel
> driver will release buffer when after used, and unstrict
> intel_iommu=on parameter will also send periodic global iotlb flush
> which will reset the whole cache. If anyone has suggestion on specific
> workload, please shoot. Anyway, I'm posting this out for review to see
> any possible comments/suggestions.
> 
> Thanks,
> 
> Peter Xu (8):
>   intel_iommu: fix VTD_PAGE_MASK
>   hmp: add info iommu
>   intel_iommu: support "info iommu"
>   intel_iommu: add iotlb/context cache statistics
>   intel_iommu: hmp: allow "-c" for "info iommu"
>   intel_iommu: let iotlb size tunable
>   intel_iommu: use access_flags for iotlb
>   intel_iommu: implement mru list for iotlb
> 
>  hmp-commands-info.hx           |  14 ++++
>  hmp.c                          |   6 ++
>  hmp.h                          |   1 +
>  hw/i386/intel_iommu.c          | 169 +++++++++++++++++++++++++++++++----------
>  hw/i386/intel_iommu_internal.h |  11 +--
>  hw/i386/trace-events           |   1 -
>  hw/i386/x86-iommu.c            |  17 +++++
>  include/hw/i386/intel_iommu.h  |  20 ++++-
>  include/hw/i386/x86-iommu.h    |   5 ++
>  include/hw/iommu.h             |   9 +++
>  stubs/Makefile.objs            |   1 +
>  stubs/iommu.c                  |   9 +++
>  12 files changed, 209 insertions(+), 54 deletions(-)
>  create mode 100644 include/hw/iommu.h
>  create mode 100644 stubs/iommu.c
> 
> -- 
> 2.7.4
> 
> 

-- 
Peter Xu

Re: [Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools

[Michael S. Tsirkin]

On Tue, Jun 27, 2017 at 05:03:31PM +0800, Peter Xu wrote:
> Patch 1: fixes a very rare PT path issue on iova value. It didn't
> break anything since it's merely not touched (only if when IOMMU
> enabled, then set one device to PT), but still better fix it.

Pls Cc HMP/QMP maintainers on interface patches.

Adding hmp interface only seems questionable to me.

Let's split this out - functionality and performance
can be separate from debugging and info.


> Patch 2-5: added "info iommu" hmp command, and implemented for VT-d.
> Meanwhile, added some statistics for iotlb.
> 
> Patch 6: introduce "x-iotlb-size" to tune iotlb size, or to turn it
> off (e.g., when we want to measure how iotlb affects one payload).
> 
> Patch 7: some refine on iotlb entry.
> 
> Patch 8: implemented MRU list algorithm for iotlb.
> 
> For the last patch, it's logically making more sense than the old
> algo, however the performance is merely the same as before (as far as
> I tested with simple netperf payloads, in either streaming, rr,
> reverse, etc.) since in most normal cases we cannot really let iotlb
> overflow especially when size is 1024 by default, e.g., guest kernel
> driver will release buffer when after used, and unstrict
> intel_iommu=on parameter will also send periodic global iotlb flush
> which will reset the whole cache. If anyone has suggestion on specific
> workload, please shoot. Anyway, I'm posting this out for review to see
> any possible comments/suggestions.
> 
> Thanks,
> 
> Peter Xu (8):
>   intel_iommu: fix VTD_PAGE_MASK
>   hmp: add info iommu
>   intel_iommu: support "info iommu"
>   intel_iommu: add iotlb/context cache statistics
>   intel_iommu: hmp: allow "-c" for "info iommu"
>   intel_iommu: let iotlb size tunable
>   intel_iommu: use access_flags for iotlb
>   intel_iommu: implement mru list for iotlb
> 
>  hmp-commands-info.hx           |  14 ++++
>  hmp.c                          |   6 ++
>  hmp.h                          |   1 +
>  hw/i386/intel_iommu.c          | 169 +++++++++++++++++++++++++++++++----------
>  hw/i386/intel_iommu_internal.h |  11 +--
>  hw/i386/trace-events           |   1 -
>  hw/i386/x86-iommu.c            |  17 +++++
>  include/hw/i386/intel_iommu.h  |  20 ++++-
>  include/hw/i386/x86-iommu.h    |   5 ++
>  include/hw/iommu.h             |   9 +++
>  stubs/Makefile.objs            |   1 +
>  stubs/iommu.c                  |   9 +++
>  12 files changed, 209 insertions(+), 54 deletions(-)
>  create mode 100644 include/hw/iommu.h
>  create mode 100644 stubs/iommu.c
> 
> -- 
> 2.7.4

Re: [Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools

[no-reply]

Hi,

This series seems to have some coding style problems. See output below for
more information:

Message-id: 1498554219-4942-1-git-send-email-peterx@redhat.com
Subject: [Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools
Type: series

=== TEST SCRIPT BEGIN ===
#!/bin/bash

BASE=base
n=1
total=$(git log --oneline $BASE.. | wc -l)
failed=0

git config --local diff.renamelimit 0
git config --local diff.renames True

commits="$(git log --format=%H --reverse $BASE..)"
for c in $commits; do
    echo "Checking PATCH $n/$total: $(git log -n 1 --format=%s $c)..."
    if ! git show $c --format=email | ./scripts/checkpatch.pl --mailback -; then
        failed=1
        echo
    fi
    n=$((n+1))
done

exit $failed
=== TEST SCRIPT END ===

Updating 3c8cf5a9c21ff8782164d1def7f44bd888713384
Switched to a new branch 'test'
1cf4f53 intel_iommu: implement mru list for iotlb
1ae1ddf intel_iommu: use access_flags for iotlb
bad3f74 intel_iommu: let iotlb size tunable
7da9464 intel_iommu: hmp: allow "-c" for "info iommu"
f2c8b10 intel_iommu: add iotlb/context cache statistics
e3a1765 intel_iommu: support "info iommu"
7bb834f hmp: add info iommu
096a817 intel_iommu: fix VTD_PAGE_MASK

=== OUTPUT BEGIN ===
Checking PATCH 1/8: intel_iommu: fix VTD_PAGE_MASK...
ERROR: Macros with complex values should be enclosed in parenthesis
#38: FILE: hw/i386/intel_iommu_internal.h:387:
+#define VTD_PAGE_MASK               ~(VTD_PAGE_SIZE - 1)

total: 1 errors, 0 warnings, 16 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.

Checking PATCH 2/8: hmp: add info iommu...
WARNING: architecture specific defines should be avoided
#79: FILE: include/hw/iommu.h:1:
+#ifndef __HW_IOMMU_H__

total: 0 errors, 1 warnings, 63 lines checked

Your patch has style problems, please review.  If any of these errors
are false positives report them to the maintainer, see
CHECKPATCH in MAINTAINERS.
Checking PATCH 3/8: intel_iommu: support "info iommu"...
Checking PATCH 4/8: intel_iommu: add iotlb/context cache statistics...
Checking PATCH 5/8: intel_iommu: hmp: allow "-c" for "info iommu"...
Checking PATCH 6/8: intel_iommu: let iotlb size tunable...
Checking PATCH 7/8: intel_iommu: use access_flags for iotlb...
Checking PATCH 8/8: intel_iommu: implement mru list for iotlb...
=== OUTPUT END ===

Test command exited with code: 1


---
Email generated automatically by Patchew [http://patchew.org/].
Please send your feedback to patchew-devel@freelists.org

Re: [Qemu-devel] [PATCH 0/8] VT-d: some enhancements on iotlb and tools

[Peter Xu]

On Tue, Jun 27, 2017 at 05:42:59PM +0300, Michael S. Tsirkin wrote:
> On Tue, Jun 27, 2017 at 05:03:31PM +0800, Peter Xu wrote:
> > Patch 1: fixes a very rare PT path issue on iova value. It didn't
> > break anything since it's merely not touched (only if when IOMMU
> > enabled, then set one device to PT), but still better fix it.
> 
> Pls Cc HMP/QMP maintainers on interface patches.
> 
> Adding hmp interface only seems questionable to me.
> 
> Let's split this out - functionality and performance
> can be separate from debugging and info.

Ok. I'll prepare separated series next time, and cc both QMP/HMP
maintainers for the interface series. Thanks,

-- 
Peter Xu