[Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

Changes since RFC:
  - use new KVM_CAP_X2APIC_API to detect x2APIC IDs support
  - rebase on top of 2.7-rc1, since many deps were merged
  - fix etc/boot-cpus to account for -device provided cpus
  - include not yet merged _PXM fix as prereq
  - add 2.8 machine type and bump up maxcpus count since it

Series extends current CPU/kvm_apic/generic pc machine
code to support x2APIC and upto 288 VCPUs when QEMU
is used with KVM's lapic.

Due to FW_CFG_MAX_CPUS (which is actually apic_id_limit)
being limited to uint16_t, the max possible APIC ID is
limitted to 2^16 with this series but that should
be sufficient for bumping VCPUs number for quite a while.

Tested with following CLI:
 QEMU -M q35 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
      -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0       \
      -bios x2apic_bios.bin


git gree for testing:
https://github.com/imammedo/qemu.git x2apic_v1

To play with the feature, one would also need x2apic enabled
seabios counterpart:
https://github.com/imammedo/seabios.git x2apic_v3

PS:
As kernel deps it needs 4.8 kernel on host side and it
doesn't include irq remapping/iommu fixes that Radim
has WIP branch, that should be posted separately/on top of this

But even without above kernel boots in x2APIC mode

Igor Mammedov (17):
  numa: reduce code duplication by adding helper numa_get_node_for_cpu()
  acpi: provide _PXM method for CPU devices if QEMU is started numa
    enabled
  tests: acpi: extend cphp testcase with numa check
  pc: acpi: x2APIC support for MADT table
  pc: acpi: x2APIC support for SRAT table
  acpi: cphp: support x2APIC entry in cpu._MAT
  acpi: cphp: force switch to modern cpu hotplug if APIC ID > 254
  pc: leave max apic_id_limit only in legacy cpu hotplug code
  pc: apic_common: extend APIC ID property to 32bit
  pc: apic_common: restore APIC ID to initial ID on reset
  pc: apic_common: reset APIC ID to initial ID when switching into
    x2APIC mode
  pc: kvm_apic: pass APIC ID depending on xAPIC/x2APIC mode
  pc: clarify FW_CFG_MAX_CPUS usage comment
  increase MAX_CPUMASK_BITS from 255 to 288
  pc: add 'etc/boot-cpus' fw_cfg file for machine with more than 255
    CPUs
  pc: add 2.8 machine
  pc: q35: bump max_cpus to 288

root (1):
  linux-headers: update to v4.8-rc1

 include/hw/acpi/acpi-defs.h        |  29 +++++++++
 include/hw/compat.h                |   2 +
 include/hw/i386/apic_internal.h    |   3 +-
 include/hw/i386/pc.h               |   4 ++
 include/sysemu/numa.h              |   3 +
 include/sysemu/sysemu.h            |   2 +-
 linux-headers/linux/kvm.h          |  13 ++++-
 target-i386/cpu.h                  |   1 +
 target-i386/kvm_i386.h             |   1 +
 hw/acpi/cpu.c                      |  17 ++++++
 hw/acpi/cpu_hotplug.c              |  17 ++++--
 hw/arm/virt-acpi-build.c           |   6 +-
 hw/arm/virt.c                      |   9 ++-
 hw/i386/acpi-build.c               | 117 +++++++++++++++++++++++++------------
 hw/i386/kvm/apic.c                 |  13 ++++-
 hw/i386/pc.c                       |  47 ++++++++-------
 hw/i386/pc_piix.c                  |  17 ++++--
 hw/i386/pc_q35.c                   |  14 ++++-
 hw/intc/apic_common.c              |  46 ++++++++++++++-
 hw/ppc/spapr.c                     |   2 +-
 hw/ppc/spapr_cpu_core.c            |   6 +-
 numa.c                             |  12 ++++
 target-i386/cpu.c                  |   2 +-
 target-i386/kvm.c                  |  14 +++++
 tests/acpi-test-data/pc/SRAT.cphp  | Bin 0 -> 304 bytes
 tests/acpi-test-data/q35/SRAT.cphp | Bin 0 -> 304 bytes
 tests/bios-tables-test.c           |   6 +-
 27 files changed, 307 insertions(+), 96 deletions(-)
 create mode 100644 tests/acpi-test-data/pc/SRAT.cphp
 create mode 100644 tests/acpi-test-data/q35/SRAT.cphp

-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 01/18] numa: reduce code duplication by adding helper numa_get_node_for_cpu()

[Igor Mammedov]

Replace repeated pattern

    for (i = 0; i < nb_numa_nodes; i++) {
        if (test_bit(idx, numa_info[i].node_cpu)) {
           ...
           break;

with a helper function to lookup numa node index for cpu.

Suggested-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: David Gibson <david@gibson.dropbear.id.au>
Reviewed-by: Shannon Zhao <shannon.zhao@linaro.org>
---
 include/sysemu/numa.h    |  3 +++
 hw/arm/virt-acpi-build.c |  6 ++----
 hw/arm/virt.c            |  7 +++----
 hw/i386/acpi-build.c     |  7 ++-----
 hw/i386/pc.c             |  8 +++-----
 hw/ppc/spapr_cpu_core.c  |  6 ++----
 numa.c                   | 12 ++++++++++++
 7 files changed, 27 insertions(+), 22 deletions(-)

diff --git a/include/sysemu/numa.h b/include/sysemu/numa.h
index bb184c9..4da808a 100644
--- a/include/sysemu/numa.h
+++ b/include/sysemu/numa.h
@@ -32,4 +32,7 @@ void numa_set_mem_node_id(ram_addr_t addr, uint64_t size, uint32_t node);
 void numa_unset_mem_node_id(ram_addr_t addr, uint64_t size, uint32_t node);
 uint32_t numa_get_node(ram_addr_t addr, Error **errp);
 
+/* on success returns node index in numa_info,
+ * on failure returns nb_numa_nodes */
+int numa_get_node_for_cpu(int idx);
 #endif
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index 28fc59c..5923b3d 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -426,11 +426,9 @@ build_srat(GArray *table_data, BIOSLinker *linker, VirtGuestInfo *guest_info)
     uint32_t *cpu_node = g_malloc0(guest_info->smp_cpus * sizeof(uint32_t));
 
     for (i = 0; i < guest_info->smp_cpus; i++) {
-        for (j = 0; j < nb_numa_nodes; j++) {
-            if (test_bit(i, numa_info[j].node_cpu)) {
+        j = numa_get_node_for_cpu(i);
+        if (j < nb_numa_nodes) {
                 cpu_node[i] = j;
-                break;
-            }
         }
     }
 
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index a193b5a..89828e5 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -413,10 +413,9 @@ static void fdt_add_cpu_nodes(const VirtBoardInfo *vbi)
                                   armcpu->mp_affinity);
         }
 
-        for (i = 0; i < nb_numa_nodes; i++) {
-            if (test_bit(cpu, numa_info[i].node_cpu)) {
-                qemu_fdt_setprop_cell(vbi->fdt, nodename, "numa-node-id", i);
-            }
+        i = numa_get_node_for_cpu(cpu);
+        if (i < nb_numa_nodes) {
+            qemu_fdt_setprop_cell(vbi->fdt, nodename, "numa-node-id", i);
         }
 
         g_free(nodename);
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index a26a4bb..3912575 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -2409,18 +2409,15 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
     srat->reserved1 = cpu_to_le32(1);
 
     for (i = 0; i < apic_ids->len; i++) {
-        int j;
+        int j = numa_get_node_for_cpu(i);
         int apic_id = apic_ids->cpus[i].arch_id;
 
         core = acpi_data_push(table_data, sizeof *core);
         core->type = ACPI_SRAT_PROCESSOR_APIC;
         core->length = sizeof(*core);
         core->local_apic_id = apic_id;
-        for (j = 0; j < nb_numa_nodes; j++) {
-            if (test_bit(i, numa_info[j].node_cpu)) {
+        if (j < nb_numa_nodes) {
                 core->proximity_lo = j;
-                break;
-            }
         }
         memset(core->proximity_hi, 0, 3);
         core->local_sapic_eid = 0;
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 47593b7..b1fadce 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -777,11 +777,9 @@ static FWCfgState *bochs_bios_init(AddressSpace *as, PCMachineState *pcms)
     for (i = 0; i < max_cpus; i++) {
         unsigned int apic_id = x86_cpu_apic_id_from_index(i);
         assert(apic_id < pcms->apic_id_limit);
-        for (j = 0; j < nb_numa_nodes; j++) {
-            if (test_bit(i, numa_info[j].node_cpu)) {
-                numa_fw_cfg[apic_id + 1] = cpu_to_le64(j);
-                break;
-            }
+        j = numa_get_node_for_cpu(i);
+        if (j < nb_numa_nodes) {
+            numa_fw_cfg[apic_id + 1] = cpu_to_le64(j);
         }
     }
     for (i = 0; i < nb_numa_nodes; i++) {
diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
index 170ed15..7b015b6 100644
--- a/hw/ppc/spapr_cpu_core.c
+++ b/hw/ppc/spapr_cpu_core.c
@@ -69,11 +69,9 @@ void spapr_cpu_init(sPAPRMachineState *spapr, PowerPCCPU *cpu, Error **errp)
     }
 
     /* Set NUMA node for the added CPUs  */
-    for (i = 0; i < nb_numa_nodes; i++) {
-        if (test_bit(cs->cpu_index, numa_info[i].node_cpu)) {
+    i = numa_get_node_for_cpu(cs->cpu_index);
+    if (i < nb_numa_nodes) {
             cs->numa_node = i;
-            break;
-        }
     }
 
     xics_cpu_setup(spapr->xics, cpu);
diff --git a/numa.c b/numa.c
index 7286171..8015f30 100644
--- a/numa.c
+++ b/numa.c
@@ -549,3 +549,15 @@ MemdevList *qmp_query_memdev(Error **errp)
     object_child_foreach(obj, query_memdev, &list);
     return list;
 }
+
+int numa_get_node_for_cpu(int idx)
+{
+    int i;
+
+    for (i = 0; i < nb_numa_nodes; i++) {
+        if (test_bit(idx, numa_info[i].node_cpu)) {
+            break;
+        }
+    }
+    return i;
+}
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 02/18] acpi: provide _PXM method for CPU devices if QEMU is started numa enabled

[Igor Mammedov]

Workaround for long standing issue where Linux kernel
assigns hotplugged CPU to 1st numa node as it discards
proximity for possible CPUs from SRAT after it's parsed.

_PXM method allows linux query proximity directly from
hotplugged CPU object, which allows Linux to assing CPU
to the correct numa node.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
---
v3:
 - use numa_get_node_for_cpu() instead of duplicating lookup code
 - added comment in code blaming linux
---
 hw/acpi/cpu.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c
index c13b65c..902f5c9 100644
--- a/hw/acpi/cpu.c
+++ b/hw/acpi/cpu.c
@@ -4,6 +4,7 @@
 #include "qapi/error.h"
 #include "qapi-event.h"
 #include "trace.h"
+#include "sysemu/numa.h"
 
 #define ACPI_CPU_HOTPLUG_REG_LEN 12
 #define ACPI_CPU_SELECTOR_OFFSET_WR 0
@@ -503,6 +504,7 @@ void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
 
         /* build Processor object for each processor */
         for (i = 0; i < arch_ids->len; i++) {
+            int j;
             Aml *dev;
             Aml *uid = aml_int(i);
             GArray *madt_buf = g_array_new(0, 1, 1);
@@ -546,6 +548,16 @@ void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
                           aml_arg(1), aml_arg(2))
             );
             aml_append(dev, method);
+
+            /* Linux guests discard SRAT info for non-present CPUs
+             * as a result _PXM is required for all CPUs which might
+             * be hot-plugged. For simplicity, add it for all CPUs.
+             */
+            j = numa_get_node_for_cpu(i);
+            if (j < nb_numa_nodes) {
+                aml_append(dev, aml_name_decl("_PXM", aml_int(j)));
+            }
+
             aml_append(cpus_dev, dev);
         }
     }
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 03/18] tests: acpi: extend cphp testcase with numa check

[Igor Mammedov]

so it would be possible to verify _PXM generation in
DSDT and SRAT tables.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
Reviewed-by: Marcel Apfelbaum <marcel@redhat.com>
---
NOTE to maintainer:
SRAT table is included in patch as it doesn't have
any chance for conflicts compared to often changing
DSDT

following table blobs should be updated in git tree as part
of this commit after running ./tests/acpi-test-data/rebuild-expected-aml.sh

tests/acpi-test-data/q35/DSDT.cphp
tests/acpi-test-data/pc/DSDT.cphp
---
 tests/acpi-test-data/pc/SRAT.cphp  | Bin 0 -> 304 bytes
 tests/acpi-test-data/q35/SRAT.cphp | Bin 0 -> 304 bytes
 tests/bios-tables-test.c           |   6 ++++--
 3 files changed, 4 insertions(+), 2 deletions(-)
 create mode 100644 tests/acpi-test-data/pc/SRAT.cphp
 create mode 100644 tests/acpi-test-data/q35/SRAT.cphp

diff --git a/tests/acpi-test-data/pc/SRAT.cphp b/tests/acpi-test-data/pc/SRAT.cphp
new file mode 100644
index 0000000000000000000000000000000000000000..ff2137642f488ec70b85207ed6c20e7351d61e98
GIT binary patch
literal 304
zcmWFzattwGWME)4bMklg2v%^42yhMtiUEZfKx_~V!f+sf!DmF1XF}yOvY_!<(fDl0
pd`1npO;83GTmZW|po75R12aq^syaB21u74tQT&BzFU&Ml8UVWm2>}2A

literal 0
HcmV?d00001

diff --git a/tests/acpi-test-data/q35/SRAT.cphp b/tests/acpi-test-data/q35/SRAT.cphp
new file mode 100644
index 0000000000000000000000000000000000000000..ff2137642f488ec70b85207ed6c20e7351d61e98
GIT binary patch
literal 304
zcmWFzattwGWME)4bMklg2v%^42yhMtiUEZfKx_~V!f+sf!DmF1XF}yOvY_!<(fDl0
pd`1npO;83GTmZW|po75R12aq^syaB21u74tQT&BzFU&Ml8UVWm2>}2A

literal 0
HcmV?d00001

diff --git a/tests/bios-tables-test.c b/tests/bios-tables-test.c
index de4019e..3796089 100644
--- a/tests/bios-tables-test.c
+++ b/tests/bios-tables-test.c
@@ -809,7 +809,8 @@ static void test_acpi_piix4_tcg_cphp(void)
     data.machine = MACHINE_PC;
     data.variant = ".cphp";
     test_acpi_one("-machine accel=tcg"
-                  " -smp 2,cores=3,sockets=2,maxcpus=6",
+                  " -smp 2,cores=3,sockets=2,maxcpus=6"
+                  " -numa node -numa node",
                   &data);
     free_test_data(&data);
 }
@@ -822,7 +823,8 @@ static void test_acpi_q35_tcg_cphp(void)
     data.machine = MACHINE_Q35;
     data.variant = ".cphp";
     test_acpi_one("-machine q35,accel=tcg"
-                  " -smp 2,cores=3,sockets=2,maxcpus=6",
+                  " -smp 2,cores=3,sockets=2,maxcpus=6"
+                  " -numa node -numa node",
                   &data);
     free_test_data(&data);
 }
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 04/18] linux-headers: update to v4.8-rc1

[Igor Mammedov]

From: root <root@kvmhaswell.lab.eng.brq.redhat.com>

Signed-off-by: Radim Krčmář <rkrcmar@redhat.com>
---
 linux-headers/linux/kvm.h | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/linux-headers/linux/kvm.h b/linux-headers/linux/kvm.h
index e60e21b..aa7f587 100644
--- a/linux-headers/linux/kvm.h
+++ b/linux-headers/linux/kvm.h
@@ -866,6 +866,10 @@ struct kvm_ppc_smmu_info {
 #define KVM_CAP_ARM_PMU_V3 126
 #define KVM_CAP_VCPU_ATTRIBUTES 127
 #define KVM_CAP_MAX_VCPU_ID 128
+#define KVM_CAP_X2APIC_API 129
+#define KVM_CAP_S390_USER_INSTR0 130
+#define KVM_CAP_MSI_DEVID 131
+#define KVM_CAP_PPC_HTM 132
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
@@ -1024,12 +1028,14 @@ struct kvm_one_reg {
 	__u64 addr;
 };
 
+#define KVM_MSI_VALID_DEVID	(1U << 0)
 struct kvm_msi {
 	__u32 address_lo;
 	__u32 address_hi;
 	__u32 data;
 	__u32 flags;
-	__u8  pad[16];
+	__u32 devid;
+	__u8  pad[12];
 };
 
 struct kvm_arm_device_addr {
@@ -1074,6 +1080,8 @@ enum kvm_device_type {
 #define KVM_DEV_TYPE_FLIC		KVM_DEV_TYPE_FLIC
 	KVM_DEV_TYPE_ARM_VGIC_V3,
 #define KVM_DEV_TYPE_ARM_VGIC_V3	KVM_DEV_TYPE_ARM_VGIC_V3
+	KVM_DEV_TYPE_ARM_VGIC_ITS,
+#define KVM_DEV_TYPE_ARM_VGIC_ITS	KVM_DEV_TYPE_ARM_VGIC_ITS
 	KVM_DEV_TYPE_MAX,
 };
 
@@ -1313,4 +1321,7 @@ struct kvm_assigned_msix_entry {
 	__u16 padding[3];
 };
 
+#define KVM_X2APIC_API_USE_32BIT_IDS            (1ULL << 0)
+#define KVM_X2APIC_API_DISABLE_BROADCAST_QUIRK  (1ULL << 1)
+
 #endif /* __LINUX_KVM_H */
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 05/18] pc: acpi: x2APIC support for MADT table

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 include/hw/acpi/acpi-defs.h | 18 +++++++++++
 hw/i386/acpi-build.c        | 78 +++++++++++++++++++++++++++++++--------------
 2 files changed, 72 insertions(+), 24 deletions(-)

diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
index 41c1d95..8f0024b 100644
--- a/include/hw/acpi/acpi-defs.h
+++ b/include/hw/acpi/acpi-defs.h
@@ -342,6 +342,24 @@ struct AcpiMadtLocalNmi {
 } QEMU_PACKED;
 typedef struct AcpiMadtLocalNmi AcpiMadtLocalNmi;
 
+struct AcpiMadtProcessorX2Apic {
+    ACPI_SUB_HEADER_DEF
+    uint16_t reserved;
+    uint32_t x2apic_id;              /* Processor's local x2APIC ID */
+    uint32_t flags;
+    uint32_t uid;                    /* Processor object _UID */
+} QEMU_PACKED;
+typedef struct AcpiMadtProcessorX2Apic AcpiMadtProcessorX2Apic;
+
+struct AcpiMadtLocalX2ApicNmi {
+    ACPI_SUB_HEADER_DEF
+    uint16_t flags;                  /* MPS INTI flags */
+    uint32_t uid;                    /* Processor object _UID */
+    uint8_t  lint;                   /* Local APIC LINT# */
+    uint8_t  reserved[3];            /* Local APIC LINT# */
+} QEMU_PACKED;
+typedef struct AcpiMadtLocalX2ApicNmi AcpiMadtLocalX2ApicNmi;
+
 struct AcpiMadtGenericInterrupt {
     ACPI_SUB_HEADER_DEF
     uint16_t reserved;
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 3912575..c5c2fbc 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -339,24 +339,38 @@ build_fadt(GArray *table_data, BIOSLinker *linker, AcpiPmInfo *pm,
 void pc_madt_cpu_entry(AcpiDeviceIf *adev, int uid,
                        CPUArchIdList *apic_ids, GArray *entry)
 {
-    int apic_id;
-    AcpiMadtProcessorApic *apic = acpi_data_push(entry, sizeof *apic);
-
-    apic_id = apic_ids->cpus[uid].arch_id;
-    apic->type = ACPI_APIC_PROCESSOR;
-    apic->length = sizeof(*apic);
-    apic->processor_id = uid;
-    apic->local_apic_id = apic_id;
-    if (apic_ids->cpus[uid].cpu != NULL) {
-        apic->flags = cpu_to_le32(1);
+    uint32_t apic_id = apic_ids->cpus[uid].arch_id;
+
+    /* ACPI spec says that LAPIC entry for non present
+     * CPU may be omitted from MADT or it must be marked
+     * as disabled. However omitting non present CPU from
+     * MADT breaks hotplug on linux. So possible CPUs
+     * should be put in MADT but kept disabled.
+     */
+    if (apic_id < 255) {
+        AcpiMadtProcessorApic *apic = acpi_data_push(entry, sizeof *apic);
+
+        apic->type = ACPI_APIC_PROCESSOR;
+        apic->length = sizeof(*apic);
+        apic->processor_id = uid;
+        apic->local_apic_id = apic_id;
+        if (apic_ids->cpus[uid].cpu != NULL) {
+            apic->flags = cpu_to_le32(1);
+        } else {
+            apic->flags = cpu_to_le32(0);
+        }
     } else {
-        /* ACPI spec says that LAPIC entry for non present
-         * CPU may be omitted from MADT or it must be marked
-         * as disabled. However omitting non present CPU from
-         * MADT breaks hotplug on linux. So possible CPUs
-         * should be put in MADT but kept disabled.
-         */
-        apic->flags = cpu_to_le32(0);
+        AcpiMadtProcessorX2Apic *apic = acpi_data_push(entry, sizeof *apic);
+
+        apic->type = ACPI_APIC_LOCAL_X2APIC;
+        apic->length = sizeof(*apic);
+        apic->uid = uid;
+        apic->x2apic_id = apic_id;
+        if (apic_ids->cpus[uid].cpu != NULL) {
+            apic->flags = cpu_to_le32(1);
+        } else {
+            apic->flags = cpu_to_le32(0);
+        }
     }
 }
 
@@ -368,11 +382,11 @@ build_madt(GArray *table_data, BIOSLinker *linker, PCMachineState *pcms)
     int madt_start = table_data->len;
     AcpiDeviceIfClass *adevc = ACPI_DEVICE_IF_GET_CLASS(pcms->acpi_dev);
     AcpiDeviceIf *adev = ACPI_DEVICE_IF(pcms->acpi_dev);
+    bool x2apic_mode = false;
 
     AcpiMultipleApicTable *madt;
     AcpiMadtIoApic *io_apic;
     AcpiMadtIntsrcovr *intsrcovr;
-    AcpiMadtLocalNmi *local_nmi;
     int i;
 
     madt = acpi_data_push(table_data, sizeof *madt);
@@ -381,6 +395,9 @@ build_madt(GArray *table_data, BIOSLinker *linker, PCMachineState *pcms)
 
     for (i = 0; i < apic_ids->len; i++) {
         adevc->madt_cpu(adev, i, apic_ids, table_data);
+        if (apic_ids->cpus[i].arch_id > 254) {
+            x2apic_mode = true;
+        }
     }
     g_free(apic_ids);
 
@@ -413,12 +430,25 @@ build_madt(GArray *table_data, BIOSLinker *linker, PCMachineState *pcms)
         intsrcovr->flags  = cpu_to_le16(0xd); /* active high, level triggered */
     }
 
-    local_nmi = acpi_data_push(table_data, sizeof *local_nmi);
-    local_nmi->type         = ACPI_APIC_LOCAL_NMI;
-    local_nmi->length       = sizeof(*local_nmi);
-    local_nmi->processor_id = 0xff; /* all processors */
-    local_nmi->flags        = cpu_to_le16(0);
-    local_nmi->lint         = 1; /* ACPI_LINT1 */
+    if (x2apic_mode) {
+        AcpiMadtLocalX2ApicNmi *local_nmi;
+
+        local_nmi = acpi_data_push(table_data, sizeof *local_nmi);
+        local_nmi->type   = ACPI_APIC_LOCAL_X2APIC_NMI;
+        local_nmi->length = sizeof(*local_nmi);
+        local_nmi->uid    = 0xFFFFFFFF; /* all processors */
+        local_nmi->flags  = cpu_to_le16(0);
+        local_nmi->lint   = 1; /* ACPI_LINT1 */
+    } else {
+        AcpiMadtLocalNmi *local_nmi;
+
+        local_nmi = acpi_data_push(table_data, sizeof *local_nmi);
+        local_nmi->type         = ACPI_APIC_LOCAL_NMI;
+        local_nmi->length       = sizeof(*local_nmi);
+        local_nmi->processor_id = 0xff; /* all processors */
+        local_nmi->flags        = cpu_to_le16(0);
+        local_nmi->lint         = 1; /* ACPI_LINT1 */
+    }
 
     build_header(linker, table_data,
                  (void *)(table_data->data + madt_start), "APIC",
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 06/18] pc: acpi: x2APIC support for SRAT table

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
rebase on top in 2.7 + updated cpu PXM patches
---
 include/hw/acpi/acpi-defs.h | 11 +++++++++++
 hw/i386/acpi-build.c        | 34 ++++++++++++++++++++++++----------
 2 files changed, 35 insertions(+), 10 deletions(-)

diff --git a/include/hw/acpi/acpi-defs.h b/include/hw/acpi/acpi-defs.h
index 8f0024b..c892704 100644
--- a/include/hw/acpi/acpi-defs.h
+++ b/include/hw/acpi/acpi-defs.h
@@ -492,6 +492,17 @@ struct AcpiSratProcessorAffinity
 } QEMU_PACKED;
 typedef struct AcpiSratProcessorAffinity AcpiSratProcessorAffinity;
 
+struct AcpiSratProcessorX2ApicAffinity {
+    ACPI_SUB_HEADER_DEF
+    uint16_t    reserved;
+    uint32_t    proximity_domain;
+    uint32_t    x2apic_id;
+    uint32_t    flags;
+    uint32_t    clk_domain;
+    uint32_t    reserved2;
+} QEMU_PACKED;
+typedef struct AcpiSratProcessorX2ApicAffinity AcpiSratProcessorX2ApicAffinity;
+
 struct AcpiSratMemoryAffinity
 {
     ACPI_SUB_HEADER_DEF
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index c5c2fbc..8cef627 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -2420,7 +2420,6 @@ static void
 build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
 {
     AcpiSystemResourceAffinityTable *srat;
-    AcpiSratProcessorAffinity *core;
     AcpiSratMemoryAffinity *numamem;
 
     int i;
@@ -2440,18 +2439,33 @@ build_srat(GArray *table_data, BIOSLinker *linker, MachineState *machine)
 
     for (i = 0; i < apic_ids->len; i++) {
         int j = numa_get_node_for_cpu(i);
-        int apic_id = apic_ids->cpus[i].arch_id;
+        uint32_t apic_id = apic_ids->cpus[i].arch_id;
 
-        core = acpi_data_push(table_data, sizeof *core);
-        core->type = ACPI_SRAT_PROCESSOR_APIC;
-        core->length = sizeof(*core);
-        core->local_apic_id = apic_id;
-        if (j < nb_numa_nodes) {
+        if (apic_id < 255) {
+            AcpiSratProcessorAffinity *core;
+
+            core = acpi_data_push(table_data, sizeof *core);
+            core->type = ACPI_SRAT_PROCESSOR_APIC;
+            core->length = sizeof(*core);
+            core->local_apic_id = apic_id;
+            if (j < nb_numa_nodes) {
                 core->proximity_lo = j;
+            }
+            memset(core->proximity_hi, 0, 3);
+            core->local_sapic_eid = 0;
+            core->flags = cpu_to_le32(1);
+        } else {
+            AcpiSratProcessorX2ApicAffinity *core;
+
+            core = acpi_data_push(table_data, sizeof *core);
+            core->type = ACPI_SRAT_PROCESSOR_x2APIC;
+            core->length = sizeof(*core);
+            core->x2apic_id = apic_id;
+            if (j < nb_numa_nodes) {
+                core->proximity_domain = cpu_to_le32(j);
+            }
+            core->flags = cpu_to_le32(1);
         }
-        memset(core->proximity_hi, 0, 3);
-        core->local_sapic_eid = 0;
-        core->flags = cpu_to_le32(1);
     }
 
 
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 07/18] acpi: cphp: support x2APIC entry in cpu._MAT

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/acpi/cpu.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/hw/acpi/cpu.c b/hw/acpi/cpu.c
index 902f5c9..5ac89fe 100644
--- a/hw/acpi/cpu.c
+++ b/hw/acpi/cpu.c
@@ -531,6 +531,11 @@ void build_cpus_aml(Aml *table, MachineState *machine, CPUHotplugFeatures opts,
                 apic->flags = cpu_to_le32(1);
                 break;
             }
+            case ACPI_APIC_LOCAL_X2APIC: {
+                AcpiMadtProcessorX2Apic *apic = (void *)madt_buf->data;
+                apic->flags = cpu_to_le32(1);
+                break;
+            }
             default:
                 assert(0);
             }
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 08/18] acpi: cphp: force switch to modern cpu hotplug if APIC ID > 254

[Igor Mammedov]

Switch to modern cpu hotplug at machine startup time if
a cpu present at boot has apic-id in range unsupported
by legacy cpu hotplug interface (i.e. > 254), to avoid
killing QEMU from legacy cpu hotplug code with error:
   "acpi: invalid cpu id: #apic-id#"

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/acpi/cpu_hotplug.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c
index e19d902..c2ab9b8 100644
--- a/hw/acpi/cpu_hotplug.c
+++ b/hw/acpi/cpu_hotplug.c
@@ -63,7 +63,8 @@ static void acpi_set_cpu_present_bit(AcpiCpuHotplug *g, CPUState *cpu,
 
     cpu_id = k->get_arch_id(cpu);
     if ((cpu_id / 8) >= ACPI_GPE_PROC_LEN) {
-        error_setg(errp, "acpi: invalid cpu id: %" PRIi64, cpu_id);
+        object_property_set_bool(g->device, false, "cpu-hotplug-legacy",
+                                 &error_abort);
         return;
     }
 
@@ -85,13 +86,14 @@ void legacy_acpi_cpu_hotplug_init(MemoryRegion *parent, Object *owner,
 {
     CPUState *cpu;
 
-    CPU_FOREACH(cpu) {
-        acpi_set_cpu_present_bit(gpe_cpu, cpu, &error_abort);
-    }
     memory_region_init_io(&gpe_cpu->io, owner, &AcpiCpuHotplug_ops,
                           gpe_cpu, "acpi-cpu-hotplug", ACPI_GPE_PROC_LEN);
     memory_region_add_subregion(parent, base, &gpe_cpu->io);
     gpe_cpu->device = owner;
+
+    CPU_FOREACH(cpu) {
+        acpi_set_cpu_present_bit(gpe_cpu, cpu, &error_abort);
+    }
 }
 
 void acpi_switch_to_modern_cphp(AcpiCpuHotplug *gpe_cpu,
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 09/18] pc: leave max apic_id_limit only in legacy cpu hotplug code

[Igor Mammedov]

that's enough to make old code that depends on it
to prevent QEMU starting with more than 255 CPUs.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/acpi/cpu_hotplug.c | 7 ++++++-
 hw/i386/pc.c          | 7 -------
 2 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/hw/acpi/cpu_hotplug.c b/hw/acpi/cpu_hotplug.c
index c2ab9b8..f15a240 100644
--- a/hw/acpi/cpu_hotplug.c
+++ b/hw/acpi/cpu_hotplug.c
@@ -15,6 +15,7 @@
 #include "qapi/error.h"
 #include "qom/cpu.h"
 #include "hw/i386/pc.h"
+#include "qemu/error-report.h"
 
 #define CPU_EJECT_METHOD "CPEJ"
 #define CPU_MAT_METHOD "CPMA"
@@ -236,7 +237,11 @@ void build_legacy_cpu_hotplug_aml(Aml *ctx, MachineState *machine,
     /* The current AML generator can cover the APIC ID range [0..255],
      * inclusive, for VCPU hotplug. */
     QEMU_BUILD_BUG_ON(ACPI_CPU_HOTPLUG_ID_LIMIT > 256);
-    g_assert(pcms->apic_id_limit <= ACPI_CPU_HOTPLUG_ID_LIMIT);
+    if (pcms->apic_id_limit > ACPI_CPU_HOTPLUG_ID_LIMIT) {
+        error_report("max_cpus is too large. APIC ID of last CPU is %u",
+                     pcms->apic_id_limit - 1);
+        exit(1);
+    }
 
     /* create PCI0.PRES device and its _CRS to reserve CPU hotplug MMIO */
     dev = aml_device("PCI0." stringify(CPU_HOTPLUG_RESOURCE_DEVICE));
diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index b1fadce..1ca5323 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -776,7 +776,6 @@ static FWCfgState *bochs_bios_init(AddressSpace *as, PCMachineState *pcms)
     numa_fw_cfg[0] = cpu_to_le64(nb_numa_nodes);
     for (i = 0; i < max_cpus; i++) {
         unsigned int apic_id = x86_cpu_apic_id_from_index(i);
-        assert(apic_id < pcms->apic_id_limit);
         j = numa_get_node_for_cpu(i);
         if (j < nb_numa_nodes) {
             numa_fw_cfg[apic_id + 1] = cpu_to_le64(j);
@@ -1188,12 +1187,6 @@ void pc_cpus_init(PCMachineState *pcms)
      * This is used for FW_CFG_MAX_CPUS. See comments on bochs_bios_init().
      */
     pcms->apic_id_limit = x86_cpu_apic_id_from_index(max_cpus - 1) + 1;
-    if (pcms->apic_id_limit > ACPI_CPU_HOTPLUG_ID_LIMIT) {
-        error_report("max_cpus is too large. APIC ID of last CPU is %u",
-                     pcms->apic_id_limit - 1);
-        exit(1);
-    }
-
     pcms->possible_cpus = g_malloc0(sizeof(CPUArchIdList) +
                                     sizeof(CPUArchId) * max_cpus);
     for (i = 0; i < max_cpus; i++) {
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 10/18] pc: apic_common: extend APIC ID property to 32bit

[Igor Mammedov]

ACPI ID is 32 bit wide on CPUs with x2APIC support.
Extend 'id' property to support it.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 include/hw/i386/apic_internal.h |  3 ++-
 target-i386/cpu.h               |  1 +
 hw/intc/apic_common.c           | 40 +++++++++++++++++++++++++++++++++++++++-
 target-i386/cpu.c               |  2 +-
 4 files changed, 43 insertions(+), 3 deletions(-)

diff --git a/include/hw/i386/apic_internal.h b/include/hw/i386/apic_internal.h
index 06c4e9f..c79b080 100644
--- a/include/hw/i386/apic_internal.h
+++ b/include/hw/i386/apic_internal.h
@@ -156,7 +156,8 @@ struct APICCommonState {
     MemoryRegion io_memory;
     X86CPU *cpu;
     uint32_t apicbase;
-    uint8_t id;
+    uint8_t id; /* legacy APIC ID */
+    uint32_t initial_apic_id;
     uint8_t version;
     uint8_t arb_id;
     uint8_t tpr;
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 65615c0..9ef4db7 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -325,6 +325,7 @@
 #define MSR_IA32_APICBASE               0x1b
 #define MSR_IA32_APICBASE_BSP           (1<<8)
 #define MSR_IA32_APICBASE_ENABLE        (1<<11)
+#define MSR_IA32_APICBASE_EXTD          (1 << 10)
 #define MSR_IA32_APICBASE_BASE          (0xfffffU<<12)
 #define MSR_IA32_FEATURE_CONTROL        0x0000003a
 #define MSR_TSC_ADJUST                  0x0000003b
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index 14ac43c..125af9d 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -21,6 +21,7 @@
 #include "qapi/error.h"
 #include "qemu-common.h"
 #include "cpu.h"
+#include "qapi/visitor.h"
 #include "hw/i386/apic.h"
 #include "hw/i386/apic_internal.h"
 #include "trace.h"
@@ -427,7 +428,6 @@ static const VMStateDescription vmstate_apic_common = {
 };
 
 static Property apic_properties_common[] = {
-    DEFINE_PROP_UINT8("id", APICCommonState, id, -1),
     DEFINE_PROP_UINT8("version", APICCommonState, version, 0x14),
     DEFINE_PROP_BIT("vapic", APICCommonState, vapic_control, VAPIC_ENABLE_BIT,
                     true),
@@ -436,6 +436,43 @@ static Property apic_properties_common[] = {
     DEFINE_PROP_END_OF_LIST(),
 };
 
+static void apic_common_get_id(Object *obj, Visitor *v, const char *name,
+                               void *opaque, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(obj);
+    int64_t value;
+
+    value = s->apicbase & MSR_IA32_APICBASE_EXTD ? s->initial_apic_id : s->id;
+    visit_type_int(v, name, &value, errp);
+}
+
+static void apic_common_set_id(Object *obj, Visitor *v, const char *name,
+                               void *opaque, Error **errp)
+{
+    APICCommonState *s = APIC_COMMON(obj);
+    Error *local_err = NULL;
+    int64_t value;
+
+    visit_type_int(v, name, &value, &local_err);
+    if (local_err) {
+        error_propagate(errp, local_err);
+        return;
+    }
+
+    s->initial_apic_id = value;
+    s->id = (uint8_t)value;
+}
+
+static void apic_common_initfn(Object *obj)
+{
+    APICCommonState *s = APIC_COMMON(obj);
+
+    s->id = s->initial_apic_id = -1;
+    object_property_add(obj, "id", "int",
+                        apic_common_get_id,
+                        apic_common_set_id, NULL, NULL, NULL);
+}
+
 static void apic_common_class_init(ObjectClass *klass, void *data)
 {
     DeviceClass *dc = DEVICE_CLASS(klass);
@@ -455,6 +492,7 @@ static const TypeInfo apic_common_type = {
     .name = TYPE_APIC_COMMON,
     .parent = TYPE_DEVICE,
     .instance_size = sizeof(APICCommonState),
+    .instance_init = apic_common_initfn,
     .class_size = sizeof(APICCommonClass),
     .class_init = apic_common_class_init,
     .abstract = true,
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 6a1afab..f7a82fe 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2830,7 +2830,7 @@ static void x86_cpu_apic_create(X86CPU *cpu, Error **errp)
                               OBJECT(cpu->apic_state), &error_abort);
     object_unref(OBJECT(cpu->apic_state));
 
-    qdev_prop_set_uint8(cpu->apic_state, "id", cpu->apic_id);
+    qdev_prop_set_uint32(cpu->apic_state, "id", cpu->apic_id);
     /* TODO: convert to link<> */
     apic = APIC_COMMON(cpu->apic_state);
     apic->cpu = cpu;
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 11/18] pc: apic_common: restore APIC ID to initial ID on reset

[Igor Mammedov]

APIC ID should be restored to initial APIC ID
state after Reset and Power-On.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/intc/apic_common.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index 125af9d..e246cf3 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -242,6 +242,7 @@ static void apic_reset_common(DeviceState *dev)
 
     bsp = s->apicbase & MSR_IA32_APICBASE_BSP;
     s->apicbase = APIC_DEFAULT_ADDRESS | bsp | MSR_IA32_APICBASE_ENABLE;
+    s->id = s->initial_apic_id;
 
     s->vapic_paddr = 0;
     info->vapic_base_update(s);
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 12/18] pc: apic_common: reset APIC ID to initial ID when switching into x2APIC mode

[Igor Mammedov]

SDM: x2APIC State Transitions:
         State Changes From xAPIC Mode to x2APIC Mode
"
Any APIC ID value written to the memory-mapped
local APIC ID register is not preserved
"

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/intc/apic_common.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
index e246cf3..492afb3 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -39,6 +39,11 @@ void cpu_set_apic_base(DeviceState *dev, uint64_t val)
     if (dev) {
         APICCommonState *s = APIC_COMMON(dev);
         APICCommonClass *info = APIC_COMMON_GET_CLASS(s);
+        /* switching to x2APIC, reset possibly modified xAPIC ID */
+        if (!(s->apicbase & MSR_IA32_APICBASE_EXTD) &&
+            (val & MSR_IA32_APICBASE_EXTD)) {
+            s->id = s->initial_apic_id;
+        }
         info->set_base(s, val);
     }
 }
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 13/18] pc: kvm_apic: pass APIC ID depending on xAPIC/x2APIC mode

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 target-i386/kvm_i386.h |  1 +
 hw/i386/kvm/apic.c     | 13 +++++++++++--
 target-i386/kvm.c      | 14 ++++++++++++++
 3 files changed, 26 insertions(+), 2 deletions(-)

diff --git a/target-i386/kvm_i386.h b/target-i386/kvm_i386.h
index 42b00af..dad0dcf 100644
--- a/target-i386/kvm_i386.h
+++ b/target-i386/kvm_i386.h
@@ -41,4 +41,5 @@ int kvm_device_msix_set_vector(KVMState *s, uint32_t dev_id, uint32_t vector,
 int kvm_device_msix_assign(KVMState *s, uint32_t dev_id);
 int kvm_device_msix_deassign(KVMState *s, uint32_t dev_id);
 
+bool kvm_has_x2apic_ids(void);
 #endif
diff --git a/hw/i386/kvm/apic.c b/hw/i386/kvm/apic.c
index 2bd0de8..3b33502 100644
--- a/hw/i386/kvm/apic.c
+++ b/hw/i386/kvm/apic.c
@@ -15,6 +15,7 @@
 #include "hw/i386/apic_internal.h"
 #include "hw/pci/msi.h"
 #include "sysemu/kvm.h"
+#include "kvm_i386.h"
 
 static inline void kvm_apic_set_reg(struct kvm_lapic_state *kapic,
                                     int reg_id, uint32_t val)
@@ -34,7 +35,11 @@ void kvm_put_apic_state(DeviceState *dev, struct kvm_lapic_state *kapic)
     int i;
 
     memset(kapic, 0, sizeof(*kapic));
-    kvm_apic_set_reg(kapic, 0x2, s->id << 24);
+    if (kvm_has_x2apic_ids() && s->apicbase & MSR_IA32_APICBASE_EXTD) {
+        kvm_apic_set_reg(kapic, 0x2, s->initial_apic_id);
+    } else {
+        kvm_apic_set_reg(kapic, 0x2, s->id << 24);
+    }
     kvm_apic_set_reg(kapic, 0x8, s->tpr);
     kvm_apic_set_reg(kapic, 0xd, s->log_dest << 24);
     kvm_apic_set_reg(kapic, 0xe, s->dest_mode << 28 | 0x0fffffff);
@@ -59,7 +64,11 @@ void kvm_get_apic_state(DeviceState *dev, struct kvm_lapic_state *kapic)
     APICCommonState *s = APIC_COMMON(dev);
     int i, v;
 
-    s->id = kvm_apic_get_reg(kapic, 0x2) >> 24;
+    if (kvm_has_x2apic_ids() && s->apicbase & MSR_IA32_APICBASE_EXTD) {
+        assert(kvm_apic_get_reg(kapic, 0x2) == s->initial_apic_id);
+    } else {
+        s->id = kvm_apic_get_reg(kapic, 0x2) >> 24;
+    }
     s->tpr = kvm_apic_get_reg(kapic, 0x8);
     s->arb_id = kvm_apic_get_reg(kapic, 0x9);
     s->log_dest = kvm_apic_get_reg(kapic, 0xd) >> 24;
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 9697e16..f1c2a93 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -111,8 +111,15 @@ static int has_pit_state2;
 
 static bool has_msr_mcg_ext_ctl;
 
+static bool has_x2apic_ids;
+
 static struct kvm_cpuid2 *cpuid_cache;
 
+bool kvm_has_x2apic_ids(void)
+{
+    return has_x2apic_ids;
+}
+
 int kvm_has_pit_state2(void)
 {
     return has_pit_state2;
@@ -1155,6 +1162,13 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
     has_pit_state2 = kvm_check_extension(s, KVM_CAP_PIT_STATE2);
 #endif
 
+#ifdef KVM_CAP_X2APIC_API
+    if (kvm_check_extension(s, KVM_CAP_X2APIC_API)) {
+        has_x2apic_ids = !kvm_vm_enable_cap(s, KVM_CAP_X2APIC_API, 0,
+                                            KVM_X2APIC_API_USE_32BIT_IDS);
+    }
+#endif
+
     ret = kvm_get_supported_msrs(s);
     if (ret < 0) {
         return ret;
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 14/18] pc: clarify FW_CFG_MAX_CPUS usage comment

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/i386/pc.c | 18 ++++++++----------
 1 file changed, 8 insertions(+), 10 deletions(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 1ca5323..c2cd5bd 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -744,17 +744,15 @@ static FWCfgState *bochs_bios_init(AddressSpace *as, PCMachineState *pcms)
 
     /* FW_CFG_MAX_CPUS is a bit confusing/problematic on x86:
      *
-     * SeaBIOS needs FW_CFG_MAX_CPUS for CPU hotplug, but the CPU hotplug
-     * QEMU<->SeaBIOS interface is not based on the "CPU index", but on the APIC
-     * ID of hotplugged CPUs[1]. This means that FW_CFG_MAX_CPUS is not the
-     * "maximum number of CPUs", but the "limit to the APIC ID values SeaBIOS
-     * may see".
+     * For machine types prior to 1.8, SeaBIOS needs FW_CFG_MAX_CPUS for
+     * building MPTable, ACPI MADT, ACPI CPU hotplug and ACPI SRAT table,
+     * that tables are based on xAPIC ID and QEMU<->SeaBIOS interface
+     * for CPU hotplug also uses APIC ID and not "CPU index".
+     * This means that FW_CFG_MAX_CPUS is not the "maximum number of CPUs",
+     * but the "limit to the APIC ID values SeaBIOS may see".
      *
-     * So, this means we must not use max_cpus, here, but the maximum possible
-     * APIC ID value, plus one.
-     *
-     * [1] The only kind of "CPU identifier" used between SeaBIOS and QEMU is
-     *     the APIC ID, not the "CPU index"
+     * So for compatibility reasons with old BIOSes we are stuck with
+     * "etc/max-cpus" actually being apic_id_limit
      */
     fw_cfg_add_i16(fw_cfg, FW_CFG_MAX_CPUS, (uint16_t)pcms->apic_id_limit);
     fw_cfg_add_i64(fw_cfg, FW_CFG_RAM_SIZE, (uint64_t)ram_size);
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 15/18] increase MAX_CPUMASK_BITS from 255 to 288

[Igor Mammedov]

so that it would be possible to increase maxcpus limit
for x86 target. Keep spapr/virt_arm at limit they used
to have 255.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 include/sysemu/sysemu.h | 2 +-
 hw/arm/virt.c           | 2 +-
 hw/ppc/spapr.c          | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index ee7c760..1250469 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -176,7 +176,7 @@ extern int mem_prealloc;
  *
  * Note that cpu->get_arch_id() may be larger than MAX_CPUMASK_BITS.
  */
-#define MAX_CPUMASK_BITS 255
+#define MAX_CPUMASK_BITS 288
 
 #define MAX_OPTION_ROMS 16
 typedef struct QEMUOptionRom {
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 89828e5..8b4e6e6 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1456,7 +1456,7 @@ static void virt_machine_class_init(ObjectClass *oc, void *data)
      * it later in machvirt_init, where we have more information about the
      * configuration of the particular instance.
      */
-    mc->max_cpus = MAX_CPUMASK_BITS;
+    mc->max_cpus = 255;
     mc->has_dynamic_sysbus = true;
     mc->block_default_type = IF_VIRTIO;
     mc->no_cdrom = 1;
diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index bce2371..fb533d1 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -2423,7 +2423,7 @@ static void spapr_machine_class_init(ObjectClass *oc, void *data)
     mc->init = ppc_spapr_init;
     mc->reset = ppc_spapr_reset;
     mc->block_default_type = IF_SCSI;
-    mc->max_cpus = MAX_CPUMASK_BITS;
+    mc->max_cpus = 255;
     mc->no_parallel = 1;
     mc->default_boot_order = "";
     mc->default_ram_size = 512 * M_BYTE;
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 16/18] pc: add 'etc/boot-cpus' fw_cfg file for machine with more than 255 CPUs

[Igor Mammedov]

Currently firmware uses 1 byte at 0x5F offset in RTC CMOS
to get number of CPUs present at boot. However 1 byte is
not enough to handle more than 255 CPUs.  So add a new
fw_cfg file that would allow QEMU to tell it.
For compat reasons add file only for machine types that
support more than 255 CPUs.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 hw/i386/pc.c | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index c2cd5bd..2b5581a 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -1235,9 +1235,11 @@ void pc_machine_done(Notifier *notifier, void *data)
     PCMachineState *pcms = container_of(notifier,
                                         PCMachineState, machine_done);
     PCIBus *bus = pcms->bus;
+    static uint16_t boot_cpus;
 
     /* set the number of CPUs */
-    rtc_set_memory(pcms->rtc, 0x5f, pc_present_cpus_count(pcms) - 1);
+    boot_cpus = pc_present_cpus_count(pcms);
+    rtc_set_memory(pcms->rtc, 0x5f, boot_cpus - 1);
 
     if (bus) {
         int extra_hosts = 0;
@@ -1258,8 +1260,16 @@ void pc_machine_done(Notifier *notifier, void *data)
 
     acpi_setup();
     if (pcms->fw_cfg) {
+        MachineClass *mc = MACHINE_GET_CLASS(pcms);
+
         pc_build_smbios(pcms->fw_cfg);
         pc_build_feature_control_file(pcms);
+
+        if (mc->max_cpus > 255) {
+            boot_cpus = cpu_to_le16(boot_cpus);
+            fw_cfg_add_file(pcms->fw_cfg, "etc/boot-cpus", &boot_cpus,
+                            sizeof(boot_cpus));
+        }
     }
 }
 
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 17/18] pc: add 2.8 machine

[Igor Mammedov]

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
 include/hw/compat.h  |  2 ++
 include/hw/i386/pc.h |  4 ++++
 hw/i386/pc_piix.c    | 16 ++++++++++++----
 hw/i386/pc_q35.c     | 13 +++++++++++--
 4 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/include/hw/compat.h b/include/hw/compat.h
index 7ee7299..779f687 100644
--- a/include/hw/compat.h
+++ b/include/hw/compat.h
@@ -1,6 +1,8 @@
 #ifndef HW_COMPAT_H
 #define HW_COMPAT_H
 
+#define HW_COMPAT_2_7
+
 #define HW_COMPAT_2_6 \
     {\
         .driver   = "virtio-mmio",\
diff --git a/include/hw/i386/pc.h b/include/hw/i386/pc.h
index 74c175c..80279dd 100644
--- a/include/hw/i386/pc.h
+++ b/include/hw/i386/pc.h
@@ -367,7 +367,11 @@ int e820_add_entry(uint64_t, uint64_t, uint32_t);
 int e820_get_num_entries(void);
 bool e820_get_entry(int, uint32_t, uint64_t *, uint64_t *);
 
+#define PC_COMPAT_2_7 \
+    HW_COMPAT_2_7
+
 #define PC_COMPAT_2_6 \
+    PC_COMPAT_2_7 \
     HW_COMPAT_2_6 \
     {\
         .driver   = "fw_cfg_io",\
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index a07dc81..d622ca8 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -438,23 +438,31 @@ static void pc_i440fx_machine_options(MachineClass *m)
     m->default_display = "std";
 }
 
-static void pc_i440fx_2_7_machine_options(MachineClass *m)
+static void pc_i440fx_2_8_machine_options(MachineClass *m)
 {
     pc_i440fx_machine_options(m);
     m->alias = "pc";
     m->is_default = 1;
 }
 
+DEFINE_I440FX_MACHINE(v2_8, "pc-i440fx-2.8", NULL,
+                      pc_i440fx_2_8_machine_options);
+
+static void pc_i440fx_2_7_machine_options(MachineClass *m)
+{
+    pc_i440fx_2_8_machine_options(m);
+    m->alias = NULL;
+    m->is_default = 0;
+    SET_MACHINE_COMPAT(m, PC_COMPAT_2_7);
+}
+
 DEFINE_I440FX_MACHINE(v2_7, "pc-i440fx-2.7", NULL,
                       pc_i440fx_2_7_machine_options);
 
-
 static void pc_i440fx_2_6_machine_options(MachineClass *m)
 {
     PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
     pc_i440fx_2_7_machine_options(m);
-    m->is_default = 0;
-    m->alias = NULL;
     pcmc->legacy_cpu_hotplug = true;
     SET_MACHINE_COMPAT(m, PC_COMPAT_2_6);
 }
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index c0b9961..9559b81 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -292,12 +292,22 @@ static void pc_q35_machine_options(MachineClass *m)
     m->has_dynamic_sysbus = true;
 }
 
-static void pc_q35_2_7_machine_options(MachineClass *m)
+static void pc_q35_2_8_machine_options(MachineClass *m)
 {
     pc_q35_machine_options(m);
     m->alias = "q35";
 }
 
+DEFINE_Q35_MACHINE(v2_8, "pc-q35-2.8", NULL,
+                   pc_q35_2_8_machine_options);
+
+static void pc_q35_2_7_machine_options(MachineClass *m)
+{
+    pc_q35_2_8_machine_options(m);
+    m->alias = NULL;
+    SET_MACHINE_COMPAT(m, PC_COMPAT_2_7);
+}
+
 DEFINE_Q35_MACHINE(v2_7, "pc-q35-2.7", NULL,
                    pc_q35_2_7_machine_options);
 
@@ -305,7 +315,6 @@ static void pc_q35_2_6_machine_options(MachineClass *m)
 {
     PCMachineClass *pcmc = PC_MACHINE_CLASS(m);
     pc_q35_2_7_machine_options(m);
-    m->alias = NULL;
     pcmc->legacy_cpu_hotplug = true;
     SET_MACHINE_COMPAT(m, PC_COMPAT_2_6);
 }
-- 
2.7.4

[Qemu-devel] [PATCH for-2.8 18/18] pc: q35: bump max_cpus to 288

[Igor Mammedov]

along with it for machine versions 2.6 and older keep
it at 255 and also freeze that limit at 255 for pc_piix4
machine as it won't be getting more than 255 CPUs.

Signed-off-by: Igor Mammedov <imammedo@redhat.com>
---
v2:
 - make 288 cpus available only since q35-2.8 machine type
---
 hw/i386/pc.c      | 2 +-
 hw/i386/pc_piix.c | 1 +
 hw/i386/pc_q35.c  | 1 +
 3 files changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/i386/pc.c b/hw/i386/pc.c
index 2b5581a..8d8f1e4 100644
--- a/hw/i386/pc.c
+++ b/hw/i386/pc.c
@@ -2294,7 +2294,7 @@ static void pc_machine_class_init(ObjectClass *oc, void *data)
     mc->query_hotpluggable_cpus = pc_query_hotpluggable_cpus;
     mc->default_boot_order = "cad";
     mc->hot_add_cpu = pc_hot_add_cpu;
-    mc->max_cpus = 255;
+    mc->max_cpus = 288;
     mc->reset = pc_machine_reset;
     hc->pre_plug = pc_machine_device_pre_plug_cb;
     hc->plug = pc_machine_device_plug_cb;
diff --git a/hw/i386/pc_piix.c b/hw/i386/pc_piix.c
index d622ca8..a383727 100644
--- a/hw/i386/pc_piix.c
+++ b/hw/i386/pc_piix.c
@@ -431,6 +431,7 @@ static void pc_xen_hvm_init(MachineState *machine)
 
 static void pc_i440fx_machine_options(MachineClass *m)
 {
+    m->max_cpus = 255;
     m->family = "pc_piix";
     m->desc = "Standard PC (i440FX + PIIX, 1996)";
     m->hot_add_cpu = pc_hot_add_cpu;
diff --git a/hw/i386/pc_q35.c b/hw/i386/pc_q35.c
index 9559b81..ef609c6 100644
--- a/hw/i386/pc_q35.c
+++ b/hw/i386/pc_q35.c
@@ -305,6 +305,7 @@ static void pc_q35_2_7_machine_options(MachineClass *m)
 {
     pc_q35_2_8_machine_options(m);
     m->alias = NULL;
+    m->max_cpus = 255;
     SET_MACHINE_COMPAT(m, PC_COMPAT_2_7);
 }
 
-- 
2.7.4

Re: [Qemu-devel] [PATCH for-2.8 16/18] pc: add 'etc/boot-cpus' fw_cfg file for machine with more than 255 CPUs

[Michael S. Tsirkin]

On Fri, Aug 05, 2016 at 11:46:15AM +0200, Igor Mammedov wrote:
> Currently firmware uses 1 byte at 0x5F offset in RTC CMOS
> to get number of CPUs present at boot. However 1 byte is
> not enough to handle more than 255 CPUs.  So add a new
> fw_cfg file that would allow QEMU to tell it.
> For compat reasons add file only for machine types that
> support more than 255 CPUs.
> 
> Signed-off-by: Igor Mammedov <imammedo@redhat.com>

Do we really need this? Why differentiate between boot
cpus and others? How about reusing the hotplug
register, such that it looks just as if all CPUs
were just added by hotplug?

> ---
>  hw/i386/pc.c | 12 +++++++++++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> index c2cd5bd..2b5581a 100644
> --- a/hw/i386/pc.c
> +++ b/hw/i386/pc.c
> @@ -1235,9 +1235,11 @@ void pc_machine_done(Notifier *notifier, void *data)
>      PCMachineState *pcms = container_of(notifier,
>                                          PCMachineState, machine_done);
>      PCIBus *bus = pcms->bus;
> +    static uint16_t boot_cpus;
>  
>      /* set the number of CPUs */
> -    rtc_set_memory(pcms->rtc, 0x5f, pc_present_cpus_count(pcms) - 1);
> +    boot_cpus = pc_present_cpus_count(pcms);
> +    rtc_set_memory(pcms->rtc, 0x5f, boot_cpus - 1);
>  
>      if (bus) {
>          int extra_hosts = 0;
> @@ -1258,8 +1260,16 @@ void pc_machine_done(Notifier *notifier, void *data)
>  
>      acpi_setup();
>      if (pcms->fw_cfg) {
> +        MachineClass *mc = MACHINE_GET_CLASS(pcms);
> +
>          pc_build_smbios(pcms->fw_cfg);
>          pc_build_feature_control_file(pcms);
> +
> +        if (mc->max_cpus > 255) {
> +            boot_cpus = cpu_to_le16(boot_cpus);
> +            fw_cfg_add_file(pcms->fw_cfg, "etc/boot-cpus", &boot_cpus,
> +                            sizeof(boot_cpus));
> +        }
>      }
>  }
>  
> -- 
> 2.7.4

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Chao Gao]

[-- Attachment #1: Type: text/plain, Size: 4994 bytes --]

HI, everyone.

We have done some tests after merging this patch set into the lastest qemu
master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
some problems we have met.

1. We can't boot up a 288 vcpus linux guest with CLI:
qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
-M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
-hda vdisk.img -device intel-iommu,intremap=on -machine q35.
The problem exists, even after we only assign 32 vcpus to the linux guest.
Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
The output of qemu and kernel is in attachments. Do you have any idea
about the problem and how to solve it?

2. Windows8.1(64bit) guest always crashes when intremap is enabled regardless
of vcpu number. We paste the crash screenshot and the log of qemu in 
attchments. We find from the qemu log that windows consistently read a 
register related to intremap before crash. what do you think cause the crash?

Regards,
-Chao

On Fri, Aug 05, 2016 at 11:45:59AM +0200, Igor Mammedov wrote:
>
>Changes since RFC:
>  - use new KVM_CAP_X2APIC_API to detect x2APIC IDs support
>  - rebase on top of 2.7-rc1, since many deps were merged
>  - fix etc/boot-cpus to account for -device provided cpus
>  - include not yet merged _PXM fix as prereq
>  - add 2.8 machine type and bump up maxcpus count since it
>
>Series extends current CPU/kvm_apic/generic pc machine
>code to support x2APIC and upto 288 VCPUs when QEMU
>is used with KVM's lapic.
>
>Due to FW_CFG_MAX_CPUS (which is actually apic_id_limit)
>being limited to uint16_t, the max possible APIC ID is
>limitted to 2^16 with this series but that should
>be sufficient for bumping VCPUs number for quite a while.
>
>Tested with following CLI:
> QEMU -M q35 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
>      -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0       \
>      -bios x2apic_bios.bin
>
>
>git gree for testing:
>https://github.com/imammedo/qemu.git x2apic_v1
>
>To play with the feature, one would also need x2apic enabled
>seabios counterpart:
>https://github.com/imammedo/seabios.git x2apic_v3
>
>PS:
>As kernel deps it needs 4.8 kernel on host side and it
>doesn't include irq remapping/iommu fixes that Radim
>has WIP branch, that should be posted separately/on top of this
>
>But even without above kernel boots in x2APIC mode
>
>Igor Mammedov (17):
>  numa: reduce code duplication by adding helper numa_get_node_for_cpu()
>  acpi: provide _PXM method for CPU devices if QEMU is started numa
>    enabled
>  tests: acpi: extend cphp testcase with numa check
>  pc: acpi: x2APIC support for MADT table
>  pc: acpi: x2APIC support for SRAT table
>  acpi: cphp: support x2APIC entry in cpu._MAT
>  acpi: cphp: force switch to modern cpu hotplug if APIC ID > 254
>  pc: leave max apic_id_limit only in legacy cpu hotplug code
>  pc: apic_common: extend APIC ID property to 32bit
>  pc: apic_common: restore APIC ID to initial ID on reset
>  pc: apic_common: reset APIC ID to initial ID when switching into
>    x2APIC mode
>  pc: kvm_apic: pass APIC ID depending on xAPIC/x2APIC mode
>  pc: clarify FW_CFG_MAX_CPUS usage comment
>  increase MAX_CPUMASK_BITS from 255 to 288
>  pc: add 'etc/boot-cpus' fw_cfg file for machine with more than 255
>    CPUs
>  pc: add 2.8 machine
>  pc: q35: bump max_cpus to 288
>
>root (1):
>  linux-headers: update to v4.8-rc1
>
> include/hw/acpi/acpi-defs.h        |  29 +++++++++
> include/hw/compat.h                |   2 +
> include/hw/i386/apic_internal.h    |   3 +-
> include/hw/i386/pc.h               |   4 ++
> include/sysemu/numa.h              |   3 +
> include/sysemu/sysemu.h            |   2 +-
> linux-headers/linux/kvm.h          |  13 ++++-
> target-i386/cpu.h                  |   1 +
> target-i386/kvm_i386.h             |   1 +
> hw/acpi/cpu.c                      |  17 ++++++
> hw/acpi/cpu_hotplug.c              |  17 ++++--
> hw/arm/virt-acpi-build.c           |   6 +-
> hw/arm/virt.c                      |   9 ++-
> hw/i386/acpi-build.c               | 117 +++++++++++++++++++++++++------------
> hw/i386/kvm/apic.c                 |  13 ++++-
> hw/i386/pc.c                       |  47 ++++++++-------
> hw/i386/pc_piix.c                  |  17 ++++--
> hw/i386/pc_q35.c                   |  14 ++++-
> hw/intc/apic_common.c              |  46 ++++++++++++++-
> hw/ppc/spapr.c                     |   2 +-
> hw/ppc/spapr_cpu_core.c            |   6 +-
> numa.c                             |  12 ++++
> target-i386/cpu.c                  |   2 +-
> target-i386/kvm.c                  |  14 +++++
> tests/acpi-test-data/pc/SRAT.cphp  | Bin 0 -> 304 bytes
> tests/acpi-test-data/q35/SRAT.cphp | Bin 0 -> 304 bytes
> tests/bios-tables-test.c           |   6 +-
> 27 files changed, 307 insertions(+), 96 deletions(-)
> create mode 100644 tests/acpi-test-data/pc/SRAT.cphp
> create mode 100644 tests/acpi-test-data/q35/SRAT.cphp
>
>-- 
>2.7.4
>
>

[-- Attachment #2: qemu_linux.log --]
[-- Type: text/plain, Size: 317653 bytes --]

***************CMD***************
qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm -M kernel-irqchip=split -serial stdio -bios /home/gao/seabios/seabios/out/bios.bin -smp cpus=288 -hda vdisk.img -device intel-iommu,intremap=on -machine q35 2>>./log/08.08.2016+13:27:53.log >&2
***************CMD***************
(vtd)vtd_register_types: 
char device redirected to /dev/pts/3 (label compat_monitor0)
Warning: Number of SMP cpus requested (288) exceeds the recommended cpus supported by KVM (240)
Warning: Number of hotpluggable cpus requested (288) exceeds the recommended cpus supported by KVM (240)
(vtd)vtd_realize: 
(vtd)vtd_reset: 
Changing serial settings was 0/0 now 3/0
SeaBIOS (version ?-20160808_104017-g.c)
BUILD: gcc: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4) binutils: version 2.23.52.0.1-55.el7 20130226
No Xen hypervisor found.
vendor 8086 device 29c0
Running on QEMU (q35)
Running on KVM
RamSize: 0x80000000 [cmos]
Relocating init from 0x000da4e0 to 0x7ffac9d0 (size 79264)
Found QEMU fw_cfg
QEMU fw_cfg DMA interface supported
RamBlock: addr 0x0000000000000000 len 0x0000000080000000 [e820]
RamBlock: addr 0x0000000100000000 len 0x0000000080000000 [e820]
Moving pm_base to 0x600
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 6 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c09f
PCI: 32: 00000000c0000000 - 00000000fec00000
PCI: map device bdf=00:02.0  bar 1, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:1f.3  bar 4, addr 0000c040, size 00000040 [io]
PCI: map device bdf=00:1f.2  bar 4, addr 0000c080, size 00000020 [io]
PCI: map device bdf=00:02.0  bar 6, addr feb80000, size 00040000 [mem]
PCI: map device bdf=00:02.0  bar 0, addr febc0000, size 00020000 [mem]
PCI: map device bdf=00:01.0  bar 6, addr febe0000, size 00010000 [mem]
PCI: map device bdf=00:01.0  bar 2, addr febf0000, size 00001000 [mem]
PCI: map device bdf=00:1f.2  bar 5, addr febf1000, size 00001000 [mem]
PCI: map device bdf=00:01.0  bar 0, addr fd000000, size 01000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:29c0
PCI: init bdf=00:01.0 id=1234:1111
PCI: init bdf=00:02.0 id=8086:100e
PCI: init bdf=00:1f.0 id=8086:2918
Q35 LPC init: elcr=00 0c
Using pmtimer, ioport 0x608
PCI: init bdf=00:1f.2 id=8086:2922
PCI: init bdf=00:1f.3 id=8086:2930
PCI: Using 00:01.0 for primary VGA
handle_smp: apic_id=121
handle_smp: apic_id=64
handle_smp: apic_id=188
handle_smp: apic_id=176
handle_smp: apic_id=84
handle_smp: apic_id=190
handle_smp: apic_id=147
handle_smp: apic_id=139
handle_smp: apic_id=179
handle_smp: apic_id=237
handle_smp: apic_id=246
handle_smp: apic_id=257
handle_smp: apic_id=109
handle_smp: apic_id=249
handle_smp: apic_id=256
handle_smp: apic_id=193
handle_smp: apic_id=105
handle_smp: apic_id=234
handle_smp: apic_id=13
handle_smp: apic_id=61
handle_smp: apic_id=141
handle_smp: apic_id=167
handle_smp: apic_id=244
handle_smp: apic_id=28
handle_smp: apic_id=106
handle_smp: apic_id=255
handle_smp: apic_id=111
handle_smp: apic_id=201
handle_smp: apic_id=42
handle_smp: apic_id=263
handle_smp: apic_id=270
handle_smp: apic_id=63
handle_smp: apic_id=275
handle_smp: apic_id=32
handle_smp: apic_id=164
handle_smp: apic_id=134
handle_smp: apic_id=41
handle_smp: apic_id=276
handle_smp: apic_id=187
handle_smp: apic_id=199
handle_smp: apic_id=20
handle_smp: apic_id=5
handle_smp: apic_id=248
handle_smp: apic_id=118
handle_smp: apic_id=66
handle_smp: apic_id=175
handle_smp: apic_id=67
handle_smp: apic_id=132
handle_smp: apic_id=46
handle_smp: apic_id=160
handle_smp: apic_id=209
handle_smp: apic_id=271
handle_smp: apic_id=181
handle_smp: apic_id=210
handle_smp: apic_id=274
handle_smp: apic_id=62
handle_smp: apic_id=196
handle_smp: apic_id=230
handle_smp: apic_id=222
handle_smp: apic_id=202
handle_smp: apic_id=131
handle_smp: apic_id=120
handle_smp: apic_id=243
handle_smp: apic_id=57
handle_smp: apic_id=95
handle_smp: apic_id=104
handle_smp: apic_id=123
handle_smp: apic_id=16
handle_smp: apic_id=24
handle_smp: apic_id=231
handle_smp: apic_id=71
handle_smp: apic_id=213
handle_smp: apic_id=278
handle_smp: apic_id=250
handle_smp: apic_id=43
handle_smp: apic_id=261
handle_smp: apic_id=144
handle_smp: apic_id=286
handle_smp: apic_id=198
handle_smp: apic_id=220
handle_smp: apic_id=37
handle_smp: apic_id=211
handle_smp: apic_id=203
handle_smp: apic_id=1
handle_smp: apic_id=137
handle_smp: apic_id=247
handle_smp: apic_id=51
handle_smp: apic_id=79
handle_smp: apic_id=242
handle_smp: apic_id=228
handle_smp: apic_id=107
handle_smp: apic_id=159
handle_smp: apic_id=254
handle_smp: apic_id=262
handle_smp: apic_id=83
handle_smp: apic_id=281
handle_smp: apic_id=232
handle_smp: apic_id=194
handle_smp: apic_id=40
handle_smp: apic_id=25
handle_smp: apic_id=170
handle_smp: apic_id=34
handle_smp: apic_id=60
handle_smp: apic_id=49
handle_smp: apic_id=148
handle_smp: apic_id=126
handle_smp: apic_id=251
handle_smp: apic_id=15
handle_smp: apic_id=86
handle_smp: apic_id=207
handle_smp: apic_id=94
handle_smp: apic_id=23
handle_smp: apic_id=205
handle_smp: apic_id=142
handle_smp: apic_id=97
handle_smp: apic_id=169
handle_smp: apic_id=285
handle_smp: apic_id=146
handle_smp: apic_id=138
handle_smp: apic_id=127
handle_smp: apic_id=65
handle_smp: apic_id=119
handle_smp: apic_id=221
handle_smp: apic_id=33
handle_smp: apic_id=149
handle_smp: apic_id=38
handle_smp: apic_id=69
handle_smp: apic_id=172
handle_smp: apic_id=130
handle_smp: apic_id=30
handle_smp: apic_id=85
handle_smp: apic_id=226
handle_smp: apic_id=154
handle_smp: apic_id=287
handle_smp: apic_id=44
handle_smp: apic_id=227
handle_smp: apic_id=9
handle_smp: apic_id=266
handle_smp: apic_id=168
handle_smp: apic_id=212
handle_smp: apic_id=214
handle_smp: apic_id=140
handle_smp: apic_id=225
handle_smp: apic_id=192
handle_smp: apic_id=189
handle_smp: apic_id=245
handle_smp: apic_id=27
handle_smp: apic_id=265
handle_smp: apic_id=200
handle_smp: apic_id=280
handle_smp: apic_id=197
handle_smp: apic_id=45
handle_smp: apic_id=76
handle_smp: apic_id=236
handle_smp: apic_id=258
handle_smp: apic_id=152
handle_smp: apic_id=78
handle_smp: apic_id=52
handle_smp: apic_id=233
handle_smp: apic_id=151
handle_smp: apic_id=177
handle_smp: apic_id=89
handle_smp: apic_id=53
handle_smp: apic_id=36
handle_smp: apic_id=56
handle_smp: apic_id=182
handle_smp: apic_id=50
handle_smp: apic_id=35
handle_smp: apic_id=163
handle_smp: apic_id=204
handle_smp: apic_id=92
handle_smp: apic_id=129
handle_smp: apic_id=161
handle_smp: apic_id=75
handle_smp: apic_id=88
handle_smp: apic_id=153
handle_smp: apic_id=12
handle_smp: apic_id=101
handle_smp: apic_id=239
handle_smp: apic_id=133
handle_smp: apic_id=238
handle_smp: apic_id=224
handle_smp: apic_id=11
handle_smp: apic_id=174
handle_smp: apic_id=136
handle_smp: apic_id=73
handle_smp: apic_id=145
handle_smp: apic_id=10
handle_smp: apic_id=272
handle_smp: apic_id=18
handle_smp: apic_id=185
handle_smp: apic_id=191
handle_smp: apic_id=235
handle_smp: apic_id=6
handle_smp: apic_id=58
handle_smp: apic_id=125
handle_smp: apic_id=240
handle_smp: apic_id=31
handle_smp: apic_id=206
handle_smp: apic_id=70
handle_smp: apic_id=91
handle_smp: apic_id=135
handle_smp: apic_id=55
handle_smp: apic_id=166
handle_smp: apic_id=113
handle_smp: apic_id=165
handle_smp: apic_id=96
handle_smp: apic_id=178
handle_smp: apic_id=93
handle_smp: apic_id=143
handle_smp: apic_id=103
handle_smp: apic_id=155
handle_smp: apic_id=124
handle_smp: apic_id=260
handle_smp: apic_id=150
handle_smp: apic_id=99
handle_smp: apic_id=21
handle_smp: apic_id=267
handle_smp: apic_id=252
handle_smp: apic_id=216
handle_smp: apic_id=116
handle_smp: apic_id=112
handle_smp: apic_id=87
handle_smp: apic_id=253
handle_smp: apic_id=90
handle_smp: apic_id=54
handle_smp: apic_id=115
handle_smp: apic_id=264
handle_smp: apic_id=122
handle_smp: apic_id=47
handle_smp: apic_id=14
handle_smp: apic_id=282
handle_smp: apic_id=241
handle_smp: apic_id=223
handle_smp: apic_id=173
handle_smp: apic_id=68
handle_smp: apic_id=208
handle_smp: apic_id=162
handle_smp: apic_id=4
handle_smp: apic_id=100
handle_smp: apic_id=74
handle_smp: apic_id=81
handle_smp: apic_id=259
handle_smp: apic_id=19
handle_smp: apic_id=218
handle_smp: apic_id=277
handle_smp: apic_id=117
handle_smp: apic_id=183
handle_smp: apic_id=283
handle_smp: apic_id=284
handle_smp: apic_id=171
handle_smp: apic_id=195
handle_smp: apic_id=2
handle_smp: apic_id=22
handle_smp: apic_id=80
handle_smp: apic_id=219
handle_smp: apic_id=229
handle_smp: apic_id=77
handle_smp: apic_id=158
handle_smp: apic_id=269
handle_smp: apic_id=273
handle_smp: apic_id=17
handle_smp: apic_id=26
handle_smp: apic_id=186
handle_smp: apic_id=180
handle_smp: apic_id=59
handle_smp: apic_id=102
handle_smp: apic_id=128
handle_smp: apic_id=110
handle_smp: apic_id=82
handle_smp: apic_id=157
handle_smp: apic_id=184
handle_smp: apic_id=7
handle_smp: apic_id=48
handle_smp: apic_id=217
handle_smp: apic_id=279
handle_smp: apic_id=98
handle_smp: apic_id=72
handle_smp: apic_id=114
handle_smp: apic_id=39
handle_smp: apic_id=29
handle_smp: apic_id=215
handle_smp: apic_id=108
handle_smp: apic_id=3
handle_smp: apic_id=156
handle_smp: apic_id=8
handle_smp: apic_id=268
Found 288 cpu(s) max supported 288 cpu(s)
Copying SMBIOS entry point from 0x00006df4 to 0x000f6be0
Scan for VGA option rom
Running option rom at c000:0003
pmm call arg1=0
Turning on vga text mode console
SeaBIOS (version ?-20160808_104017-g.c)
AHCI controller at 00:1f.2, iobase 0xfebf1000, irq 10
Searching bootorder for: /pci@i0cf8/*@1f,2/drive@0/disk@0
AHCI/0: Set transfer mode to UDMA-5
AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk (10240 MiBytes)"
Searching bootorder for: /pci@i0cf8/*@1f,2/drive@2/disk@0
AHCI/2: registering: "DVD/CD [AHCI/2: QEMU DVD-ROM ATAPI-4 DVD/CD]"
PS2 keyboard initialized
Found 1 lpt ports
Found 1 serial ports
All threads complete.
Scan for option roms
Running option rom at c980:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /pci@i0cf8/*@2
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f6b40: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=20971520
Running option rom at ca80:0003
Space available for UMB: cd000-ec000, f6400-f6ac0
Returned 106496 bytes of ZoneHigh
e820 map has 10 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 000000007ffda000 = 1 RAM
  4: 000000007ffda000 - 0000000080000000 = 2 RESERVED
  5: 00000000b0000000 - 00000000c0000000 = 2 RESERVED
  6: 00000000fed1c000 - 00000000fed20000 = 2 RESERVED
  7: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
  8: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
  9: 0000000100000000 - 0000000180000000 = 1 RAM
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00
(vtd)vtd_mem_read: addr 0x8 size 4 val 0x22260206
(vtd)vtd_mem_read: addr 0xc size 4 val 0x12008c
(vtd)vtd_mem_read: addr 0x10 size 4 val 0xf00f1a
(vtd)vtd_mem_read: addr 0x14 size 4 val 0x0
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.10.0-327.el7.x86_64 (mockbuild@x86-034.build.eng.bos.redhat.com) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Oct 29 17:29:29 EDT 2015
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb console=ttyS0,115200 nosoftlockup LANG=en_US.UTF-8 intel_iommu=on
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ffd9fff] usable
[    0.000000] BIOS-e820: [mem 0x000000007ffda000-0x000000007fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.8 present.
[    0.000000] Hypervisor detected: KVM
[    0.000000] AGP: No AGP bridge found
[    0.000000] e820: last_pfn = 0x180000 max_arch_pfn = 0x400000000
[    0.000000] PAT not supported by CPU.
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] e820: last_pfn = 0x7ffda max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[    0.000000] init_memory_mapping: [mem 0x17fe00000-0x17fffffff]
[    0.000000] init_memory_mapping: [mem 0x17c000000-0x17fdfffff]
[    0.000000] init_memory_mapping: [mem 0x100000000-0x17bffffff]
[    0.000000] init_memory_mapping: [mem 0x00100000-0x7ffd9fff]
[    0.000000] RAMDISK: [mem 0x35a4c000-0x36d1dfff]
[    0.000000] ACPI: RSDP 00000000000f6bb0 00014 (v00 BOCHS )
[    0.000000] ACPI: RSDT 000000007ffe47a2 00038 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACP 000000007ffe3bf4 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.000000] ACPI: DSDT 000000007ffdb580 08674 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACS 000000007ffdb540 00040
[    0.000000] ACPI: APIC 000000007ffe3c68 00A7E (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.000000] ACPI: HPET 000000007ffe46e6 00038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.000000] ACPI: MCFG 000000007ffe471e 0003C (v01 BOCHS  BXPCMCFG 00000001 BXPC 00000001)
[    0.000000] ACPI: DMAR 000000007ffe475a 00048 (v01 BOCHS  BXPCDMAR 00000001 BXPC 00000001)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000017fffffff]
[    0.000000] Initmem setup node 0 [mem 0x00000000-0x17fffffff]
[    0.000000]   NODE_DATA [mem 0x17ffd7000-0x17fffdfff]
[    0.000000] Reserving 161MB of memory at 688MB for crashkernel (System RAM: 4095MB)
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 1:7ff87001, primary cpu clock
[    0.000000] kvm-clock: using sched offset of 10313837607 cycles
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
[    0.000000]   Normal   [mem 0x100000000-0x17fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x0009efff]
[    0.000000]   node   0: [mem 0x00100000-0x7ffd9fff]
[    0.000000]   node   0: [mem 0x100000000-0x17fffffff]
[    0.000000] ACPI: PM-Timer IO Port: 0x608
[    0.000000] ACPI: X2APIC (apic_id[0xff] uid[0xff] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x100] uid[0x100] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x101] uid[0x101] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x102] uid[0x102] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x103] uid[0x103] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x104] uid[0x104] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x105] uid[0x105] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x106] uid[0x106] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x107] uid[0x107] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x108] uid[0x108] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x109] uid[0x109] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10a] uid[0x10a] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10b] uid[0x10b] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10c] uid[0x10c] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10d] uid[0x10d] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10e] uid[0x10e] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10f] uid[0x10f] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x110] uid[0x110] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x111] uid[0x111] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x112] uid[0x112] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x113] uid[0x113] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x114] uid[0x114] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x115] uid[0x115] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x116] uid[0x116] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x117] uid[0x117] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x118] uid[0x118] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x119] uid[0x119] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11a] uid[0x11a] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11b] uid[0x11b] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11c] uid[0x11c] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11d] uid[0x11d] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11e] uid[0x11e] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11f] uid[0x11f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x10] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x11] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x12] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x13] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x14] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x15] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x16] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x17] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x18] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x19] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x1a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x1d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x1e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x1f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x20] lapic_id[0x20] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x21] lapic_id[0x21] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x22] lapic_id[0x22] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x23] lapic_id[0x23] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x24] lapic_id[0x24] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x25] lapic_id[0x25] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x26] lapic_id[0x26] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x27] lapic_id[0x27] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x28] lapic_id[0x28] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x29] lapic_id[0x29] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2a] lapic_id[0x2a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2b] lapic_id[0x2b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2c] lapic_id[0x2c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2d] lapic_id[0x2d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2e] lapic_id[0x2e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2f] lapic_id[0x2f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x30] lapic_id[0x30] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x31] lapic_id[0x31] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x32] lapic_id[0x32] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x33] lapic_id[0x33] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x34] lapic_id[0x34] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x35] lapic_id[0x35] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x36] lapic_id[0x36] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x37] lapic_id[0x37] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x38] lapic_id[0x38] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x39] lapic_id[0x39] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3a] lapic_id[0x3a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3b] lapic_id[0x3b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3c] lapic_id[0x3c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3d] lapic_id[0x3d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3e] lapic_id[0x3e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3f] lapic_id[0x3f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x40] lapic_id[0x40] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x41] lapic_id[0x41] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x42] lapic_id[0x42] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x43] lapic_id[0x43] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x44] lapic_id[0x44] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x45] lapic_id[0x45] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x46] lapic_id[0x46] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x47] lapic_id[0x47] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x48] lapic_id[0x48] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x49] lapic_id[0x49] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4a] lapic_id[0x4a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4b] lapic_id[0x4b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4c] lapic_id[0x4c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4d] lapic_id[0x4d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4e] lapic_id[0x4e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4f] lapic_id[0x4f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x50] lapic_id[0x50] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x51] lapic_id[0x51] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x52] lapic_id[0x52] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x53] lapic_id[0x53] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x54] lapic_id[0x54] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x55] lapic_id[0x55] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x56] lapic_id[0x56] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x57] lapic_id[0x57] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x58] lapic_id[0x58] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x59] lapic_id[0x59] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5a] lapic_id[0x5a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5b] lapic_id[0x5b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5c] lapic_id[0x5c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5d] lapic_id[0x5d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5e] lapic_id[0x5e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5f] lapic_id[0x5f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x60] lapic_id[0x60] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x61] lapic_id[0x61] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x62] lapic_id[0x62] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x63] lapic_id[0x63] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x64] lapic_id[0x64] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x65] lapic_id[0x65] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x66] lapic_id[0x66] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x67] lapic_id[0x67] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x68] lapic_id[0x68] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x69] lapic_id[0x69] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6a] lapic_id[0x6a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6b] lapic_id[0x6b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6c] lapic_id[0x6c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6d] lapic_id[0x6d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6e] lapic_id[0x6e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6f] lapic_id[0x6f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x70] lapic_id[0x70] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x71] lapic_id[0x71] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x72] lapic_id[0x72] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x73] lapic_id[0x73] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x74] lapic_id[0x74] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x75] lapic_id[0x75] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x76] lapic_id[0x76] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x77] lapic_id[0x77] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x78] lapic_id[0x78] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x79] lapic_id[0x79] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7a] lapic_id[0x7a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7b] lapic_id[0x7b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7c] lapic_id[0x7c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7d] lapic_id[0x7d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7e] lapic_id[0x7e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7f] lapic_id[0x7f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x80] lapic_id[0x80] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x81] lapic_id[0x81] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x82] lapic_id[0x82] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x83] lapic_id[0x83] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x84] lapic_id[0x84] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x85] lapic_id[0x85] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x86] lapic_id[0x86] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x87] lapic_id[0x87] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x88] lapic_id[0x88] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x89] lapic_id[0x89] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8a] lapic_id[0x8a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8b] lapic_id[0x8b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8c] lapic_id[0x8c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8d] lapic_id[0x8d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8e] lapic_id[0x8e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8f] lapic_id[0x8f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x90] lapic_id[0x90] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x91] lapic_id[0x91] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x92] lapic_id[0x92] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x93] lapic_id[0x93] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x94] lapic_id[0x94] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x95] lapic_id[0x95] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x96] lapic_id[0x96] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x97] lapic_id[0x97] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x98] lapic_id[0x98] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x99] lapic_id[0x99] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9a] lapic_id[0x9a] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9b] lapic_id[0x9b] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9c] lapic_id[0x9c] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9d] lapic_id[0x9d] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9e] lapic_id[0x9e] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9f] lapic_id[0x9f] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa0] lapic_id[0xa0] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa1] lapic_id[0xa1] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa2] lapic_id[0xa2] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa3] lapic_id[0xa3] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa4] lapic_id[0xa4] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa5] lapic_id[0xa5] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa6] lapic_id[0xa6] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa7] lapic_id[0xa7] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa8] lapic_id[0xa8] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa9] lapic_id[0xa9] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xaa] lapic_id[0xaa] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xab] lapic_id[0xab] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xac] lapic_id[0xac] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xad] lapic_id[0xad] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xae] lapic_id[0xae] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xaf] lapic_id[0xaf] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb0] lapic_id[0xb0] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb1] lapic_id[0xb1] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb2] lapic_id[0xb2] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb3] lapic_id[0xb3] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb4] lapic_id[0xb4] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb5] lapic_id[0xb5] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb6] lapic_id[0xb6] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb7] lapic_id[0xb7] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb8] lapic_id[0xb8] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb9] lapic_id[0xb9] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xba] lapic_id[0xba] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbb] lapic_id[0xbb] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbc] lapic_id[0xbc] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbd] lapic_id[0xbd] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbe] lapic_id[0xbe] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbf] lapic_id[0xbf] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc0] lapic_id[0xc0] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc1] lapic_id[0xc1] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc2] lapic_id[0xc2] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc3] lapic_id[0xc3] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc4] lapic_id[0xc4] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc5] lapic_id[0xc5] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc6] lapic_id[0xc6] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc7] lapic_id[0xc7] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc8] lapic_id[0xc8] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc9] lapic_id[0xc9] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xca] lapic_id[0xca] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcb] lapic_id[0xcb] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcc] lapic_id[0xcc] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcd] lapic_id[0xcd] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xce] lapic_id[0xce] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcf] lapic_id[0xcf] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd0] lapic_id[0xd0] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd1] lapic_id[0xd1] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd2] lapic_id[0xd2] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd3] lapic_id[0xd3] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd4] lapic_id[0xd4] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd5] lapic_id[0xd5] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd6] lapic_id[0xd6] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd7] lapic_id[0xd7] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd8] lapic_id[0xd8] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd9] lapic_id[0xd9] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xda] lapic_id[0xda] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdb] lapic_id[0xdb] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdc] lapic_id[0xdc] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdd] lapic_id[0xdd] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xde] lapic_id[0xde] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdf] lapic_id[0xdf] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe0] lapic_id[0xe0] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe1] lapic_id[0xe1] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe2] lapic_id[0xe2] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe3] lapic_id[0xe3] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe4] lapic_id[0xe4] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe5] lapic_id[0xe5] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe6] lapic_id[0xe6] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe7] lapic_id[0xe7] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe8] lapic_id[0xe8] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe9] lapic_id[0xe9] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xea] lapic_id[0xea] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xeb] lapic_id[0xeb] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xec] lapic_id[0xec] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xed] lapic_id[0xed] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xee] lapic_id[0xee] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xef] lapic_id[0xef] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf0] lapic_id[0xf0] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf1] lapic_id[0xf1] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf2] lapic_id[0xf2] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf3] lapic_id[0xf3] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf4] lapic_id[0xf4] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf5] lapic_id[0xf5] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf6] lapic_id[0xf6] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf7] lapic_id[0xf7] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf8] lapic_id[0xf8] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf9] lapic_id[0xf9] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfa] lapic_id[0xfa] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfb] lapic_id[0xfb] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfc] lapic_id[0xfc] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfd] lapic_id[0xfd] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfe] lapic_id[0xfe] enabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] dfl dfl lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] smpboot: Allowing 288 CPUs, 0 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x7ffda000-0x7fffffff]
[    0.000000] PM: Registered nosave memory: [mem 0x80000000-0xafffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xb0000000-0xbfffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfed1bfff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed20000-0xfeffbfff]
[    0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
[    0.000000] e820: [mem 0xc0000000-0xfed1bfff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:288 nr_cpu_ids:288 nr_node_ids:1
[    0.000000] PERCPU: Embedded 31 pages/cpu @ffff880179200000 s87168 r8192 d31616 u131072
[    0.000000] KVM setup async PF for cpu 0
[    0.000000] kvm-stealtime: cpu 0, msr 17920d240
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1032035
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb console=ttyS0,115200 nosoftlockup LANG=en_US.UTF-8 intel_iommu=on
[    0.000000] Intel-IOMMU: enabled
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] AGP: Checking aperture...
[    0.000000] AGP: No AGP bridge found
[    0.000000] Memory: 3825916k/6291456k available (6440k kernel code, 2097696k absent, 367844k reserved, 4266k data, 1620k init)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=288, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=288.
[    0.000000] 	Offload RCU callbacks from all CPUs
[    0.000000] 	Offload RCU callbacks from CPUs: 0-287.
[    0.000000] NR_IRQS:327936 nr_irqs:2728 0
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] allocated 16777216 bytes of page_cgroup
[    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[    0.000000] tsc: Detected 3591.682 MHz processor
[    0.993720] Calibrating delay loop (skipped) preset value.. 7183.36 BogoMIPS (lpj=3591682)
[    0.997166] pid_max: default: 294912 minimum: 2304
[    0.999234] Security Framework initialized
[    1.000965] SELinux:  Initializing.
[    1.002858] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    1.006816] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    1.010431] Mount-cache hash table entries: 4096
[    1.012864] Initializing cgroup subsys memory
[    1.014768] Initializing cgroup subsys devices
[    1.016765] Initializing cgroup subsys freezer
[    1.018709] Initializing cgroup subsys net_cls
[    1.020726] Initializing cgroup subsys blkio
[    1.022785] Initializing cgroup subsys perf_event
[    1.025091] Initializing cgroup subsys hugetlb
[    1.027375] CPU: Physical Processor ID: 0
[    1.029276] mce: CPU supports 10 MCE banks
[    1.031285] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    1.031285] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
[    1.031285] tlb_flushall_shift: 6
[    1.037730] Freeing SMP alternatives: 28k freed
[    1.042167] ACPI: Core revision 20130517
[    1.047626] ACPI: All ACPI Tables successfully acquired
[    1.049930] ftrace: allocating 24585 entries in 97 pages
[    1.071011] dmar: Host address width 39
[    1.072764] dmar: DRHD base: 0x000000fed90000 flags: 0x1
(vtd)vtd_mem_read: addr 0x8 size 4 val 0x22260206
(vtd)vtd_mem_read: addr 0xc size 4 val 0x12008c
(vtd)vtd_mem_read: addr 0x10 size 4 val 0xf00f1a
(vtd)vtd_mem_read: addr 0x14 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x0 size 4 val 0x10
[    1.075322] dmar: IOMMU 0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
[    1.080072] IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
[    1.082378] Queued invalidation will be enabled to support x2apic and Intr-remapping.
(vtd)vtd_mem_read: addr 0x34 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_read: addr 0x20 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_read: addr 0x20 size 4 val 0x0
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x4000000
(vtd)vtd_handle_gcmd_write: value 0x4000000 status 0x0
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x4000000
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x5000000
(vtd)vtd_handle_gcmd_write: value 0x5000000 status 0x4000000
(vtd)vtd_handle_gcmd_sirtp: set Interrupt Remap Table Pointer
(vtd)vtd_interrupt_remap_table_setup: int remap table addr 0x177100000 size 65536
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x5000000
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x7000000
(vtd)vtd_handle_gcmd_write: value 0x7000000 status 0x5000000
(vtd)vtd_handle_gcmd_sirtp: set Interrupt Remap Table Pointer
(vtd)vtd_interrupt_remap_table_setup: int remap table addr 0x177100000 size 65536
(vtd)vtd_handle_gcmd_ire: Interrupt Remap Enable on
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x7000000
[    1.085926] Enabled IRQ remapping in x2apic mode
(vtd)vtd_mem_read: addr 0x38 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x3c size 4 val 0x4141
(vtd)vtd_mem_read: addr 0x40 size 4 val 0xfee0100c
(vtd)vtd_mem_read: addr 0x44 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x34 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x34 size 4 val 0x0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 48, index: 1
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 48, index: 1
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 1
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 1
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 1
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 0, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[    1.100983] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    1.103720] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (fam: 06, model: 06, stepping: 03)
[    1.209759] Performance Events: Broken PMU hardware detected, using software events only.
[    1.214098] Failed to access perfctr msr (MSR c2 is 0)
[    1.221330] smpboot: Booting Node   0, Processors  #1[    1.234651] kvm-clock: cpu 1, msr 1:7ff87041, secondary cpu clock
[    1.257702] KVM setup async PF for cpu 1
 #2[    1.259826] kvm-stealtime: cpu 1, msr 17922d240
[    1.268956] kvm-clock: cpu 2, msr 1:7ff87081, secondary cpu clock
[    1.293046] KVM setup async PF for cpu 2
 #3[    1.295740] kvm-stealtime: cpu 2, msr 17924d240
[    1.304738] kvm-clock: cpu 3, msr 1:7ff870c1, secondary cpu clock
[    1.328652] KVM setup async PF for cpu 3
 #4[    1.331157] kvm-stealtime: cpu 3, msr 17926d240
[    1.340631] kvm-clock: cpu 4, msr 1:7ff87101, secondary cpu clock
[    1.365008] KVM setup async PF for cpu 4
 #5[    1.367768] kvm-stealtime: cpu 4, msr 17928d240
[    1.376731] kvm-clock: cpu 5, msr 1:7ff87141, secondary cpu clock
[    1.401717] KVM setup async PF for cpu 5
 #6[    1.404751] kvm-stealtime: cpu 5, msr 1792ad240
[    1.413792] kvm-clock: cpu 6, msr 1:7ff87181, secondary cpu clock
[    1.439185] KVM setup async PF for cpu 6
 #7[    1.442845] kvm-stealtime: cpu 6, msr 1792cd240
[    1.451060] kvm-clock: cpu 7, msr 1:7ff871c1, secondary cpu clock
[    1.476317] KVM setup async PF for cpu 7
 #8[    1.480146] kvm-stealtime: cpu 7, msr 1792ed240
[    1.488043] kvm-clock: cpu 8, msr 1:7ff87201, secondary cpu clock
[    1.512978] KVM setup async PF for cpu 8
 #9[    1.516231] kvm-stealtime: cpu 8, msr 17930d240
[    1.525415] kvm-clock: cpu 9, msr 1:7ff87241, secondary cpu clock
[    1.551219] KVM setup async PF for cpu 9
 #10[    1.554999] kvm-stealtime: cpu 9, msr 17932d240
[    1.563163] kvm-clock: cpu 10, msr 1:7ff87281, secondary cpu clock
[    1.589143] KVM setup async PF for cpu 10
 #11[    1.593201] kvm-stealtime: cpu 10, msr 17934d240
[    1.600887] kvm-clock: cpu 11, msr 1:7ff872c1, secondary cpu clock
[    1.627219] KVM setup async PF for cpu 11
 #12[    1.631422] kvm-stealtime: cpu 11, msr 17936d240
[    1.639350] kvm-clock: cpu 12, msr 1:7ff87301, secondary cpu clock
[    1.666757] KVM setup async PF for cpu 12
 #13[    1.670735] kvm-stealtime: cpu 12, msr 17938d240
[    1.680259] kvm-clock: cpu 13, msr 1:7ff87341, secondary cpu clock
[    1.704998] KVM setup async PF for cpu 13
 #14[    1.710484] kvm-stealtime: cpu 13, msr 1793ad240
[    1.717213] kvm-clock: cpu 14, msr 1:7ff87381, secondary cpu clock
[    1.742572] KVM setup async PF for cpu 14
 #15[    1.748140] kvm-stealtime: cpu 14, msr 1793cd240
[    1.754142] kvm-clock: cpu 15, msr 1:7ff873c1, secondary cpu clock
[    1.783945] KVM setup async PF for cpu 15
[    1.795505] kvm-clock: cpu 16, msr 1:7ff87401, secondary cpu clock
 #16[    1.819359] kvm-stealtime: cpu 15, msr 1793ed240
 #17[    1.828181] KVM setup async PF for cpu 16
[    1.828183] kvm-stealtime: cpu 16, msr 17940d240
[    1.850025] kvm-clock: cpu 17, msr 1:7ff87441, secondary cpu clock
[    1.881278] KVM setup async PF for cpu 17
 #18[    1.886044] kvm-stealtime: cpu 17, msr 17942d240
[    1.894346] kvm-clock: cpu 18, msr 1:7ff87481, secondary cpu clock
[    1.920829] KVM setup async PF for cpu 18
[    1.932725] kvm-clock: cpu 19, msr 1:7ff874c1, secondary cpu clock
 #19[    1.961539] kvm-stealtime: cpu 18, msr 17944d240
 #20[    1.976705] KVM setup async PF for cpu 19
[    1.976707] kvm-stealtime: cpu 19, msr 17946d240
[    1.999704] kvm-clock: cpu 20, msr 1:7ff87501, secondary cpu clock
[    2.033586] KVM setup async PF for cpu 20
 #21[    2.044299] kvm-stealtime: cpu 20, msr 17948d240
[    2.045836] kvm-clock: cpu 21, msr 1:7ff87541, secondary cpu clock
[    2.091017] KVM setup async PF for cpu 21
 #22[    2.100550] kvm-stealtime: cpu 21, msr 1794ad240
[    2.102838] kvm-clock: cpu 22, msr 1:7ff87581, secondary cpu clock
[    2.145192] KVM setup async PF for cpu 22
 #23[    2.153568] kvm-stealtime: cpu 22, msr 1794cd240
[    2.157988] kvm-clock: cpu 23, msr 1:7ff875c1, secondary cpu clock
[    2.197523] KVM setup async PF for cpu 23
[    2.208945] kvm-clock: cpu 24, msr 1:7ff87601, secondary cpu clock
 #24[    2.248282] kvm-stealtime: cpu 23, msr 1794ed240
[    2.256994] KVM setup async PF for cpu 24
 #25[    2.266289] kvm-stealtime: cpu 24, msr 17950d240
[    2.269536] kvm-clock: cpu 25, msr 1:7ff87641, secondary cpu clock
[    2.304223] KVM setup async PF for cpu 25
 #26[    2.314136] kvm-stealtime: cpu 25, msr 17952d240
[    2.316454] kvm-clock: cpu 26, msr 1:7ff87681, secondary cpu clock
[    2.357423] KVM setup async PF for cpu 26
[    2.368980] kvm-clock: cpu 27, msr 1:7ff876c1, secondary cpu clock
 #27[    2.407995] kvm-stealtime: cpu 26, msr 17954d240
[    2.420826] KVM setup async PF for cpu 27
[    2.433370] kvm-clock: cpu 28, msr 1:7ff87701, secondary cpu clock
 #28[    2.471827] kvm-stealtime: cpu 27, msr 17956d240
[    2.488320] KVM setup async PF for cpu 28
[    2.500588] kvm-clock: cpu 29, msr 1:7ff87741, secondary cpu clock
 #29[    2.533349] kvm-stealtime: cpu 28, msr 17958d240
[    2.554217] KVM setup async PF for cpu 29
[    2.566332] kvm-clock: cpu 30, msr 1:7ff87781, secondary cpu clock
 #30[    2.602892] kvm-stealtime: cpu 29, msr 1795ad240
 #31[    2.624975] KVM setup async PF for cpu 30
[    2.624977] kvm-stealtime: cpu 30, msr 1795cd240
[    2.652889] kvm-clock: cpu 31, msr 1:7ff877c1, secondary cpu clock
[    2.693346] KVM setup async PF for cpu 31
 #32[    2.704715] kvm-stealtime: cpu 31, msr 1795ed240
[    2.706733] kvm-clock: cpu 32, msr 1:7ff87801, secondary cpu clock
[    2.756351] KVM setup async PF for cpu 32
[    2.768769] kvm-clock: cpu 33, msr 1:7ff87841, secondary cpu clock
 #33[    2.807531] kvm-stealtime: cpu 32, msr 17960d240
[    2.818798] KVM setup async PF for cpu 33
[    2.832476] kvm-clock: cpu 34, msr 1:7ff87881, secondary cpu clock
 #34[    2.972188] kvm-stealtime: cpu 33, msr 17962d240
[    2.991678] KVM setup async PF for cpu 34
[    3.003758] kvm-clock: cpu 35, msr 1:7ff878c1, secondary cpu clock
 #35[    3.034710] kvm-stealtime: cpu 34, msr 17964d240
 #36[    3.056635] KVM setup async PF for cpu 35
[    3.067911] kvm-clock: cpu 36, msr 1:7ff87901, secondary cpu clock
[    3.104980] kvm-stealtime: cpu 35, msr 17966d240
[    3.120440] KVM setup async PF for cpu 36
[    3.133968] kvm-clock: cpu 37, msr 1:7ff87941, secondary cpu clock
 #37[    3.168489] kvm-stealtime: cpu 36, msr 17968d240
[    3.184873] KVM setup async PF for cpu 37
[    3.197469] kvm-clock: cpu 38, msr 1:7ff87981, secondary cpu clock
 #38[    3.230123] kvm-stealtime: cpu 37, msr 1796ad240
[    3.241759] KVM setup async PF for cpu 38
[    3.253848] kvm-clock: cpu 39, msr 1:7ff879c1, secondary cpu clock
 #39[    3.295292] kvm-stealtime: cpu 38, msr 1796cd240
[    3.306317] KVM setup async PF for cpu 39
[    3.319591] kvm-clock: cpu 40, msr 1:7ff87a01, secondary cpu clock
 #40[    3.364138] kvm-stealtime: cpu 39, msr 1796ed240
[    3.383284] KVM setup async PF for cpu 40
[    3.396310] kvm-clock: cpu 41, msr 1:7ff87a41, secondary cpu clock
 #41[    3.434347] kvm-stealtime: cpu 40, msr 17970d240
[    3.455016] KVM setup async PF for cpu 41
[    3.467640] kvm-clock: cpu 42, msr 1:7ff87a81, secondary cpu clock
 #42[    3.503519] kvm-stealtime: cpu 41, msr 17972d240
 #43[    3.520948] KVM setup async PF for cpu 42
[    3.520950] kvm-stealtime: cpu 42, msr 17974d240
[    3.551508] kvm-clock: cpu 43, msr 1:7ff87ac1, secondary cpu clock
[    3.584161] KVM setup async PF for cpu 43
 #44[    3.594195] kvm-stealtime: cpu 43, msr 17976d240
[    3.598116] kvm-clock: cpu 44, msr 1:7ff87b01, secondary cpu clock
[    3.663669] KVM setup async PF for cpu 44
 #45[    3.674506] kvm-stealtime: cpu 44, msr 17978d240
[    3.677567] kvm-clock: cpu 45, msr 1:7ff87b41, secondary cpu clock
[    3.735990] KVM setup async PF for cpu 45
[    3.748620] kvm-clock: cpu 46, msr 1:7ff87b81, secondary cpu clock
 #46[    3.793250] kvm-stealtime: cpu 45, msr 1797ad240
[    3.818908] KVM setup async PF for cpu 46
[    3.831465] kvm-clock: cpu 47, msr 1:7ff87bc1, secondary cpu clock
 #47[    3.873304] kvm-stealtime: cpu 46, msr 1797cd240
[    3.893749] KVM setup async PF for cpu 47
 #48[    3.904314] kvm-stealtime: cpu 47, msr 1797ed240
[    3.907813] kvm-clock: cpu 48, msr 1:7ff87c01, secondary cpu clock
[    3.977182] KVM setup async PF for cpu 48
[    3.989840] kvm-clock: cpu 49, msr 1:7ff87c41, secondary cpu clock
 #49[    4.118552] kvm-stealtime: cpu 48, msr 17980d240
[    4.145917] KVM setup async PF for cpu 49
[    4.158336] kvm-clock: cpu 50, msr 1:7ff87c81, secondary cpu clock
 #50[    4.200079] kvm-stealtime: cpu 49, msr 17982d240
[    4.234271] KVM setup async PF for cpu 50
[    4.247464] kvm-clock: cpu 51, msr 1:7ff87cc1, secondary cpu clock
 #51[    4.289511] kvm-stealtime: cpu 50, msr 17984d240
[    4.312522] KVM setup async PF for cpu 51
[    4.325511] kvm-clock: cpu 52, msr 1:7ff87d01, secondary cpu clock
 #52[    4.371908] kvm-stealtime: cpu 51, msr 17986d240
[    4.396411] KVM setup async PF for cpu 52
[    4.409521] kvm-clock: cpu 53, msr 1:7ff87d41, secondary cpu clock
 #53[    4.446267] kvm-stealtime: cpu 52, msr 17988d240
[    4.470871] KVM setup async PF for cpu 53
 #54[    4.482827] kvm-stealtime: cpu 53, msr 1798ad240
[    4.484035] kvm-clock: cpu 54, msr 1:7ff87d81, secondary cpu clock
[    4.553538] KVM setup async PF for cpu 54
[    4.566058] kvm-clock: cpu 55, msr 1:7ff87dc1, secondary cpu clock
 #55[    4.606410] kvm-stealtime: cpu 54, msr 1798cd240
 #56[    4.649927] KVM setup async PF for cpu 55
[    4.649930] kvm-stealtime: cpu 55, msr 1798ed240
[    4.688879] kvm-clock: cpu 56, msr 1:7ff87e01, secondary cpu clock
[    4.730035] KVM setup async PF for cpu 56
 #57[    4.744277] kvm-stealtime: cpu 56, msr 17990d240
[    4.745599] kvm-clock: cpu 57, msr 1:7ff87e41, secondary cpu clock
[    4.800388] KVM setup async PF for cpu 57
[    4.816037] kvm-clock: cpu 58, msr 1:7ff87e81, secondary cpu clock
 #58[    4.861840] kvm-stealtime: cpu 57, msr 17992d240
 #59[    4.879355] KVM setup async PF for cpu 58
[    4.879357] kvm-stealtime: cpu 58, msr 17994d240
[    4.922791] kvm-clock: cpu 59, msr 1:7ff87ec1, secondary cpu clock
[    4.972236] KVM setup async PF for cpu 59
 #60[    4.985798] kvm-stealtime: cpu 59, msr 17996d240
[    4.985811] kvm-clock: cpu 60, msr 1:7ff87f01, secondary cpu clock
 #61[    5.058976] KVM setup async PF for cpu 60
[    5.058978] kvm-stealtime: cpu 60, msr 17998d240
[    5.101302] kvm-clock: cpu 61, msr 1:7ff87f41, secondary cpu clock
[    5.137491] KVM setup async PF for cpu 61
[    5.150608] kvm-clock: cpu 62, msr 1:7ff87f81, secondary cpu clock
 #62[    5.230002] kvm-stealtime: cpu 61, msr 1799ad240
[    5.289343] KVM setup async PF for cpu 62
 #63[    5.315572] kvm-stealtime: cpu 62, msr 1799cd240
[    5.315636] kvm-clock: cpu 63, msr 1:7ff87fc1, secondary cpu clock
[    5.465363] KVM setup async PF for cpu 63
[    5.480638] kvm-clock: cpu 64, msr 1:7ff88001, secondary cpu clock
 #64[    5.519408] kvm-stealtime: cpu 63, msr 1799ed240
[    5.549705] KVM setup async PF for cpu 64
[    5.562841] kvm-clock: cpu 65, msr 1:7ff88041, secondary cpu clock
 #65[    5.614272] kvm-stealtime: cpu 64, msr 179a0d240
[    5.642321] KVM setup async PF for cpu 65
[    5.656027] kvm-clock: cpu 66, msr 1:7ff88081, secondary cpu clock
 #66[    5.720372] kvm-stealtime: cpu 65, msr 179a2d240
 #67[    5.747785] KVM setup async PF for cpu 66
[    5.747788] kvm-stealtime: cpu 66, msr 179a4d240
[    5.794898] kvm-clock: cpu 67, msr 1:7ff880c1, secondary cpu clock
[    5.838775] KVM setup async PF for cpu 67
[    5.854046] kvm-clock: cpu 68, msr 1:7ff88101, secondary cpu clock
 #68[    5.895628] kvm-stealtime: cpu 67, msr 179a6d240
[    5.918008] KVM setup async PF for cpu 68
[    5.934151] kvm-clock: cpu 69, msr 1:7ff88141, secondary cpu clock
 #69[    6.000275] kvm-stealtime: cpu 68, msr 179a8d240
[    6.027911] KVM setup async PF for cpu 69
[    6.041850] kvm-clock: cpu 70, msr 1:7ff88181, secondary cpu clock
 #70[    6.100225] kvm-stealtime: cpu 69, msr 179aad240
[    6.122558] KVM setup async PF for cpu 70
[    6.140071] kvm-clock: cpu 71, msr 1:7ff881c1, secondary cpu clock
 #71[    6.195706] kvm-stealtime: cpu 70, msr 179acd240
[    6.231386] KVM setup async PF for cpu 71
[    6.246234] kvm-clock: cpu 72, msr 1:7ff88201, secondary cpu clock
 #72[    6.299394] kvm-stealtime: cpu 71, msr 179aed240
 #73[    6.323455] KVM setup async PF for cpu 72
[    6.323458] kvm-stealtime: cpu 72, msr 179b0d240
[    6.372543] kvm-clock: cpu 73, msr 1:7ff88241, secondary cpu clock
[    6.420285] KVM setup async PF for cpu 73
[    6.433187] kvm-clock: cpu 74, msr 1:7ff88281, secondary cpu clock
 #74[    6.492934] kvm-stealtime: cpu 73, msr 179b2d240
[    6.528891] KVM setup async PF for cpu 74
[    6.541563] kvm-clock: cpu 75, msr 1:7ff882c1, secondary cpu clock
 #75[    6.598101] kvm-stealtime: cpu 74, msr 179b4d240
[    6.623063] KVM setup async PF for cpu 75
[    6.642029] kvm-clock: cpu 76, msr 1:7ff88301, secondary cpu clock
 #76[    6.698128] kvm-stealtime: cpu 75, msr 179b6d240
[    6.720013] KVM setup async PF for cpu 76
[    6.735310] kvm-clock: cpu 77, msr 1:7ff88341, secondary cpu clock
 #77[    6.913906] kvm-stealtime: cpu 76, msr 179b8d240
[    6.962949] KVM setup async PF for cpu 77
[    6.976414] kvm-clock: cpu 78, msr 1:7ff88381, secondary cpu clock
 #78[    7.045495] kvm-stealtime: cpu 77, msr 179bad240
 #79[    7.074910] KVM setup async PF for cpu 78
[    7.074913] kvm-stealtime: cpu 78, msr 179bcd240
[    7.122740] kvm-clock: cpu 79, msr 1:7ff883c1, secondary cpu clock
[    7.175252] KVM setup async PF for cpu 79
[    7.190528] kvm-clock: cpu 80, msr 1:7ff88401, secondary cpu clock
 #80[    7.236898] kvm-stealtime: cpu 79, msr 179bed240
[    7.261661] KVM setup async PF for cpu 80
[    7.275576] kvm-clock: cpu 81, msr 1:7ff88441, secondary cpu clock
 #81[    7.340793] kvm-stealtime: cpu 80, msr 179c0d240
[    7.364637] KVM setup async PF for cpu 81
[    7.383907] kvm-clock: cpu 82, msr 1:7ff88481, secondary cpu clock
 #82[    7.432319] kvm-stealtime: cpu 81, msr 179c2d240
 #83[    7.464377] KVM setup async PF for cpu 82
[    7.464380] kvm-stealtime: cpu 82, msr 179c4d240
[    7.515172] kvm-clock: cpu 83, msr 1:7ff884c1, secondary cpu clock
[    7.563280] KVM setup async PF for cpu 83
[    7.579204] kvm-clock: cpu 84, msr 1:7ff88501, secondary cpu clock
 #84[    7.630637] kvm-stealtime: cpu 83, msr 179c6d240
[    7.661417] KVM setup async PF for cpu 84
[    7.679397] kvm-clock: cpu 85, msr 1:7ff88541, secondary cpu clock
 #85[    7.745093] kvm-stealtime: cpu 84, msr 179c8d240
[    7.773895] KVM setup async PF for cpu 85
[    7.794637] kvm-clock: cpu 86, msr 1:7ff88581, secondary cpu clock
 #86[    7.850158] kvm-stealtime: cpu 85, msr 179cad240
[    7.882057] KVM setup async PF for cpu 86
[    7.898825] kvm-clock: cpu 87, msr 1:7ff885c1, secondary cpu clock
 #87[    7.966599] kvm-stealtime: cpu 86, msr 179ccd240
[    8.006126] KVM setup async PF for cpu 87
[    8.023581] kvm-clock: cpu 88, msr 1:7ff88601, secondary cpu clock
 #88[    8.069758] kvm-stealtime: cpu 87, msr 179ced240
[    8.102526] KVM setup async PF for cpu 88
[    8.115979] kvm-clock: cpu 89, msr 1:7ff88641, secondary cpu clock
 #89[    8.163658] kvm-stealtime: cpu 88, msr 179d0d240
[    8.199680] KVM setup async PF for cpu 89
[    8.213390] kvm-clock: cpu 90, msr 1:7ff88681, secondary cpu clock
 #90[    8.286418] kvm-stealtime: cpu 89, msr 179d2d240
[    8.312047] KVM setup async PF for cpu 90
[    8.328238] kvm-clock: cpu 91, msr 1:7ff886c1, secondary cpu clock
 #91[    8.377995] kvm-stealtime: cpu 90, msr 179d4d240
[    8.413506] KVM setup async PF for cpu 91
 #92[    8.433202] kvm-stealtime: cpu 91, msr 179d6d240
[    8.439088] kvm-clock: cpu 92, msr 1:7ff88701, secondary cpu clock
 #93[    8.514138] KVM setup async PF for cpu 92
[    8.514140] kvm-stealtime: cpu 92, msr 179d8d240
[    8.575344] kvm-clock: cpu 93, msr 1:7ff88741, secondary cpu clock
[    8.628418] KVM setup async PF for cpu 93
 #94[    8.648581] kvm-stealtime: cpu 93, msr 179dad240
[    8.648586] kvm-clock: cpu 94, msr 1:7ff88781, secondary cpu clock
[    8.722736] KVM setup async PF for cpu 94
[    8.738697] kvm-clock: cpu 95, msr 1:7ff887c1, secondary cpu clock
 #95[    8.806987] kvm-stealtime: cpu 94, msr 179dcd240
[    8.834767] KVM setup async PF for cpu 95
[    8.850544] kvm-clock: cpu 96, msr 1:7ff88801, secondary cpu clock
 #96[    8.912908] kvm-stealtime: cpu 95, msr 179ded240
 #97[    8.954216] KVM setup async PF for cpu 96
[    8.954218] kvm-stealtime: cpu 96, msr 179e0d240
[    9.010098] kvm-clock: cpu 97, msr 1:7ff88841, secondary cpu clock
[    9.062270] KVM setup async PF for cpu 97
[    9.076598] kvm-clock: cpu 98, msr 1:7ff88881, secondary cpu clock
 #98[    9.138432] kvm-stealtime: cpu 97, msr 179e2d240
[    9.172071] KVM setup async PF for cpu 98
[    9.189354] kvm-clock: cpu 99, msr 1:7ff888c1, secondary cpu clock
 #99[    9.244059] kvm-stealtime: cpu 98, msr 179e4d240
[    9.274917] KVM setup async PF for cpu 99
[    9.290848] kvm-clock: cpu 100, msr 1:7ff88901, secondary cpu clock
 #100[    9.344862] kvm-stealtime: cpu 99, msr 179e6d240
[    9.391662] KVM setup async PF for cpu 100
[    9.415580] kvm-clock: cpu 101, msr 1:7ff88941, secondary cpu clock
 #101[    9.489014] kvm-stealtime: cpu 100, msr 179e8d240
[    9.522028] KVM setup async PF for cpu 101
[    9.539007] kvm-clock: cpu 102, msr 1:7ff88981, secondary cpu clock
 #102[    9.838009] kvm-stealtime: cpu 101, msr 179ead240
[    9.878255] KVM setup async PF for cpu 102
[    9.893689] kvm-clock: cpu 103, msr 1:7ff889c1, secondary cpu clock
 #103[    9.974875] kvm-stealtime: cpu 102, msr 179ecd240
[   10.017670] KVM setup async PF for cpu 103
[   10.033038] kvm-clock: cpu 104, msr 1:7ff88a01, secondary cpu clock
 #104[   10.101403] kvm-stealtime: cpu 103, msr 179eed240
[   10.131355] KVM setup async PF for cpu 104
[   10.146444] kvm-clock: cpu 105, msr 1:7ff88a41, secondary cpu clock
 #105[   10.218639] kvm-stealtime: cpu 104, msr 179f0d240
[   10.261702] KVM setup async PF for cpu 105
[   10.277533] kvm-clock: cpu 106, msr 1:7ff88a81, secondary cpu clock
 #106[   10.473141] kvm-stealtime: cpu 105, msr 179f2d240
[   10.515512] KVM setup async PF for cpu 106
[   10.538503] kvm-clock: cpu 107, msr 1:7ff88ac1, secondary cpu clock
 #107[   10.583279] kvm-stealtime: cpu 106, msr 179f4d240
[   10.617792] KVM setup async PF for cpu 107
[   10.640125] kvm-clock: cpu 108, msr 1:7ff88b01, secondary cpu clock
 #108[   10.693047] kvm-stealtime: cpu 107, msr 179f6d240
[   10.733161] KVM setup async PF for cpu 108
[   10.749696] kvm-clock: cpu 109, msr 1:7ff88b41, secondary cpu clock
 #109[   10.799434] kvm-stealtime: cpu 108, msr 179f8d240
 #110[   10.839743] KVM setup async PF for cpu 109
[   10.839746] kvm-stealtime: cpu 109, msr 179fad240
[   10.910442] kvm-clock: cpu 110, msr 1:7ff88b81, secondary cpu clock
[   10.984631] KVM setup async PF for cpu 110
[   11.003814] kvm-clock: cpu 111, msr 1:7ff88bc1, secondary cpu clock
 #111[   11.059304] kvm-stealtime: cpu 110, msr 179fcd240
[   11.106249] KVM setup async PF for cpu 111
[   11.123648] kvm-clock: cpu 112, msr 1:7ff88c01, secondary cpu clock
 #112[   11.215281] kvm-stealtime: cpu 111, msr 179fed240
[   11.249131] KVM setup async PF for cpu 112
[   11.262128] kvm-clock: cpu 113, msr 1:7ff88c41, secondary cpu clock
 #113[   11.456075] kvm-stealtime: cpu 112, msr 17a00d240
 #114[   11.496012] KVM setup async PF for cpu 113
[   11.496015] kvm-stealtime: cpu 113, msr 17a02d240
[   11.565996] kvm-clock: cpu 114, msr 1:7ff88c81, secondary cpu clock
[   11.637306] KVM setup async PF for cpu 114
[   11.654390] kvm-clock: cpu 115, msr 1:7ff88cc1, secondary cpu clock
 #115[   11.729973] kvm-stealtime: cpu 114, msr 17a04d240
[   11.777467] KVM setup async PF for cpu 115
[   11.791311] kvm-clock: cpu 116, msr 1:7ff88d01, secondary cpu clock
 #116[   12.037660] kvm-stealtime: cpu 115, msr 17a06d240
[   12.098038] KVM setup async PF for cpu 116
[   12.111839] kvm-clock: cpu 117, msr 1:7ff88d41, secondary cpu clock
 #117[   12.204824] kvm-stealtime: cpu 116, msr 17a08d240
[   12.246465] KVM setup async PF for cpu 117
[   12.263908] kvm-clock: cpu 118, msr 1:7ff88d81, secondary cpu clock
 #118[   12.503917] kvm-stealtime: cpu 117, msr 17a0ad240
[   12.541654] KVM setup async PF for cpu 118
[   12.562295] kvm-clock: cpu 119, msr 1:7ff88dc1, secondary cpu clock
 #119[   12.629453] kvm-stealtime: cpu 118, msr 17a0cd240
 #120[   12.675042] KVM setup async PF for cpu 119
[   12.675048] kvm-stealtime: cpu 119, msr 17a0ed240
[   12.747699] kvm-clock: cpu 120, msr 1:7ff88e01, secondary cpu clock
[   12.826186] KVM setup async PF for cpu 120
[   12.841452] kvm-clock: cpu 121, msr 1:7ff88e41, secondary cpu clock
 #121[   12.918485] kvm-stealtime: cpu 120, msr 17a10d240
 #122[   12.957830] KVM setup async PF for cpu 121
[   12.957834] kvm-stealtime: cpu 121, msr 17a12d240
[   13.034204] kvm-clock: cpu 122, msr 1:7ff88e81, secondary cpu clock
[   13.099189] KVM setup async PF for cpu 122
[   13.115586] kvm-clock: cpu 123, msr 1:7ff88ec1, secondary cpu clock
 #123[   13.205791] kvm-stealtime: cpu 122, msr 17a14d240
[   13.322608] KVM setup async PF for cpu 123
[   13.340099] kvm-clock: cpu 124, msr 1:7ff88f01, secondary cpu clock
 #124[   13.415549] kvm-stealtime: cpu 123, msr 17a16d240
[   13.457814] KVM setup async PF for cpu 124
[   13.483744] kvm-clock: cpu 125, msr 1:7ff88f41, secondary cpu clock
 #125[   13.666835] kvm-stealtime: cpu 124, msr 17a18d240
 #126[   13.741449] KVM setup async PF for cpu 125
[   13.741451] kvm-stealtime: cpu 125, msr 17a1ad240
[   13.810603] kvm-clock: cpu 126, msr 1:7ff88f81, secondary cpu clock
[   13.874046] KVM setup async PF for cpu 126
[   13.893688] kvm-clock: cpu 127, msr 1:7ff88fc1, secondary cpu clock
 #127[   13.983975] kvm-stealtime: cpu 126, msr 17a1cd240
[   14.027697] KVM setup async PF for cpu 127
[   14.054376] kvm-clock: cpu 128, msr 1:7ff89001, secondary cpu clock
 #128[   14.216168] kvm-stealtime: cpu 127, msr 17a1ed240
[   14.250880] KVM setup async PF for cpu 128
[   14.269375] kvm-clock: cpu 129, msr 1:7ff89041, secondary cpu clock
 #129[   14.354993] kvm-stealtime: cpu 128, msr 17a20d240
[   14.397981] KVM setup async PF for cpu 129
[   14.410709] kvm-clock: cpu 130, msr 1:7ff89081, secondary cpu clock
 #130[   14.507130] kvm-stealtime: cpu 129, msr 17a22d240
[   14.557448] KVM setup async PF for cpu 130
[   14.576076] kvm-clock: cpu 131, msr 1:7ff890c1, secondary cpu clock
 #131[   14.663255] kvm-stealtime: cpu 130, msr 17a24d240
[   14.708037] KVM setup async PF for cpu 131
[   14.732191] kvm-clock: cpu 132, msr 1:7ff89101, secondary cpu clock
 #132[   14.811789] kvm-stealtime: cpu 131, msr 17a26d240
[   14.855815] KVM setup async PF for cpu 132
[   14.876480] kvm-clock: cpu 133, msr 1:7ff89141, secondary cpu clock
 #133[   14.974508] kvm-stealtime: cpu 132, msr 17a28d240
 #134[   15.036181] KVM setup async PF for cpu 133
[   15.036184] kvm-stealtime: cpu 133, msr 17a2ad240
[   15.118277] kvm-clock: cpu 134, msr 1:7ff89181, secondary cpu clock
[   15.195018] KVM setup async PF for cpu 134
[   15.213629] kvm-clock: cpu 135, msr 1:7ff891c1, secondary cpu clock
 #135[   15.287922] kvm-stealtime: cpu 134, msr 17a2cd240
[   15.327647] KVM setup async PF for cpu 135
[   15.343552] kvm-clock: cpu 136, msr 1:7ff89201, secondary cpu clock
 #136[   15.419431] kvm-stealtime: cpu 135, msr 17a2ed240
 #137[   15.466993] KVM setup async PF for cpu 136
[   15.466996] kvm-stealtime: cpu 136, msr 17a30d240
[   15.554957] kvm-clock: cpu 137, msr 1:7ff89241, secondary cpu clock
[   15.625787] KVM setup async PF for cpu 137
[   15.644529] kvm-clock: cpu 138, msr 1:7ff89281, secondary cpu clock
 #138[   15.715860] kvm-stealtime: cpu 137, msr 17a32d240
[   15.780078] KVM setup async PF for cpu 138
[   15.799577] kvm-clock: cpu 139, msr 1:7ff892c1, secondary cpu clock
 #139[   15.912742] kvm-stealtime: cpu 138, msr 17a34d240
[   15.980039] KVM setup async PF for cpu 139
[   15.999337] kvm-clock: cpu 140, msr 1:7ff89301, secondary cpu clock
 #140[   16.115081] kvm-stealtime: cpu 139, msr 17a36d240
[   16.164350] KVM setup async PF for cpu 140
[   16.189662] kvm-clock: cpu 141, msr 1:7ff89341, secondary cpu clock
 #141[   16.265723] kvm-stealtime: cpu 140, msr 17a38d240
[   16.312216] KVM setup async PF for cpu 141
[   16.335617] kvm-clock: cpu 142, msr 1:7ff89381, secondary cpu clock
 #142[   16.424070] kvm-stealtime: cpu 141, msr 17a3ad240
[   16.488774] KVM setup async PF for cpu 142
[   16.506714] kvm-clock: cpu 143, msr 1:7ff893c1, secondary cpu clock
 #143[   16.596548] kvm-stealtime: cpu 142, msr 17a3cd240
[   16.990164] KVM setup async PF for cpu 143
[   17.008058] kvm-clock: cpu 144, msr 1:7ff89401, secondary cpu clock
 #144[   17.093867] kvm-stealtime: cpu 143, msr 17a3ed240
[   17.137632] KVM setup async PF for cpu 144
[   17.156041] kvm-clock: cpu 145, msr 1:7ff89441, secondary cpu clock
 #145[   17.272614] kvm-stealtime: cpu 144, msr 17a40d240
[   17.323900] KVM setup async PF for cpu 145
[   17.342983] kvm-clock: cpu 146, msr 1:7ff89481, secondary cpu clock
 #146[   17.448833] kvm-stealtime: cpu 145, msr 17a42d240
[   17.505340] KVM setup async PF for cpu 146
[   17.519284] kvm-clock: cpu 147, msr 1:7ff894c1, secondary cpu clock
 #147[   17.617780] kvm-stealtime: cpu 146, msr 17a44d240
[   17.677766] KVM setup async PF for cpu 147
[   17.708334] kvm-clock: cpu 148, msr 1:7ff89501, secondary cpu clock
 #148[   17.779824] kvm-stealtime: cpu 147, msr 17a46d240
[   17.822007] KVM setup async PF for cpu 148
[   17.840771] kvm-clock: cpu 149, msr 1:7ff89541, secondary cpu clock
 #149[   17.921766] kvm-stealtime: cpu 148, msr 17a48d240
[   17.969199] KVM setup async PF for cpu 149
[   17.984038] kvm-clock: cpu 150, msr 1:7ff89581, secondary cpu clock
 #150[   18.081917] kvm-stealtime: cpu 149, msr 17a4ad240
[   18.130253] KVM setup async PF for cpu 150
[   18.148223] kvm-clock: cpu 151, msr 1:7ff895c1, secondary cpu clock
 #151[   18.225125] kvm-stealtime: cpu 150, msr 17a4cd240
[   18.273957] KVM setup async PF for cpu 151
[   18.292526] kvm-clock: cpu 152, msr 1:7ff89601, secondary cpu clock
 #152[   18.380707] kvm-stealtime: cpu 151, msr 17a4ed240
[   18.536784] KVM setup async PF for cpu 152
[   18.554010] kvm-clock: cpu 153, msr 1:7ff89641, secondary cpu clock
 #153[   18.661818] kvm-stealtime: cpu 152, msr 17a50d240
[   18.715424] KVM setup async PF for cpu 153
[   18.739890] kvm-clock: cpu 154, msr 1:7ff89681, secondary cpu clock
 #154[   18.811488] kvm-stealtime: cpu 153, msr 17a52d240
[   18.876989] KVM setup async PF for cpu 154
[   18.903758] kvm-clock: cpu 155, msr 1:7ff896c1, secondary cpu clock
 #155[   18.973934] kvm-stealtime: cpu 154, msr 17a54d240
[   19.016052] KVM setup async PF for cpu 155
[   19.042895] kvm-clock: cpu 156, msr 1:7ff89701, secondary cpu clock
 #156[   19.123329] kvm-stealtime: cpu 155, msr 17a56d240
[   19.180411] KVM setup async PF for cpu 156
[   19.205861] kvm-clock: cpu 157, msr 1:7ff89741, secondary cpu clock
 #157[   19.282121] kvm-stealtime: cpu 156, msr 17a58d240
[   19.337983] KVM setup async PF for cpu 157
[   19.354570] kvm-clock: cpu 158, msr 1:7ff89781, secondary cpu clock
 #158[   19.455163] kvm-stealtime: cpu 157, msr 17a5ad240
[   19.508468] KVM setup async PF for cpu 158
[   19.524660] kvm-clock: cpu 159, msr 1:7ff897c1, secondary cpu clock
 #159[   19.615593] kvm-stealtime: cpu 158, msr 17a5cd240
 #160[   19.668018] KVM setup async PF for cpu 159
[   19.668020] kvm-stealtime: cpu 159, msr 17a5ed240
[   19.779853] kvm-clock: cpu 160, msr 1:7ff89801, secondary cpu clock
[   19.865993] KVM setup async PF for cpu 160
[   19.883009] kvm-clock: cpu 161, msr 1:7ff89841, secondary cpu clock
 #161[   19.984156] kvm-stealtime: cpu 160, msr 17a60d240
[   20.039415] KVM setup async PF for cpu 161
[   20.188870] kvm-clock: cpu 162, msr 1:7ff89881, secondary cpu clock
 #162[   20.449782] kvm-stealtime: cpu 161, msr 17a62d240
[   20.532367] KVM setup async PF for cpu 162
[   20.549045] kvm-clock: cpu 163, msr 1:7ff898c1, secondary cpu clock
 #163[   20.675870] kvm-stealtime: cpu 162, msr 17a64d240
[   20.746450] KVM setup async PF for cpu 163
[   20.760735] kvm-clock: cpu 164, msr 1:7ff89901, secondary cpu clock
 #164[   20.878151] kvm-stealtime: cpu 163, msr 17a66d240
[   21.057631] KVM setup async PF for cpu 164
[   21.087835] kvm-clock: cpu 165, msr 1:7ff89941, secondary cpu clock
 #165[   21.183644] kvm-stealtime: cpu 164, msr 17a68d240
[   21.248308] KVM setup async PF for cpu 165
[   21.279239] kvm-clock: cpu 166, msr 1:7ff89981, secondary cpu clock
 #166[   21.385360] kvm-stealtime: cpu 165, msr 17a6ad240
[   21.438092] KVM setup async PF for cpu 166
[   21.462356] kvm-clock: cpu 167, msr 1:7ff899c1, secondary cpu clock
 #167[   21.567972] kvm-stealtime: cpu 166, msr 17a6cd240
[   21.621723] KVM setup async PF for cpu 167
[   21.640716] kvm-clock: cpu 168, msr 1:7ff89a01, secondary cpu clock
 #168[   21.750538] kvm-stealtime: cpu 167, msr 17a6ed240
[   21.802678] KVM setup async PF for cpu 168
[   21.823823] kvm-clock: cpu 169, msr 1:7ff89a41, secondary cpu clock
 #169[   21.929144] kvm-stealtime: cpu 168, msr 17a70d240
 #170[   21.990518] KVM setup async PF for cpu 169
[   21.990546] kvm-stealtime: cpu 169, msr 17a72d240
[   22.085157] kvm-clock: cpu 170, msr 1:7ff89a81, secondary cpu clock
[   22.163564] KVM setup async PF for cpu 170
[   22.180989] kvm-clock: cpu 171, msr 1:7ff89ac1, secondary cpu clock
 #171[   22.292520] kvm-stealtime: cpu 170, msr 17a74d240
[   22.346449] KVM setup async PF for cpu 171
[   22.360790] kvm-clock: cpu 172, msr 1:7ff89b01, secondary cpu clock
 #172[   22.461349] kvm-stealtime: cpu 171, msr 17a76d240
[   22.521585] KVM setup async PF for cpu 172
[   22.544888] kvm-clock: cpu 173, msr 1:7ff89b41, secondary cpu clock
 #173[   22.640179] kvm-stealtime: cpu 172, msr 17a78d240
[   22.688924] KVM setup async PF for cpu 173
[   22.714153] kvm-clock: cpu 174, msr 1:7ff89b81, secondary cpu clock
[   22.799130] kvm-stealtime: cpu 173, msr 17a7ad240
[   22.692496]  #174 #175[   22.841034] KVM setup async PF for cpu 174
[   22.841037] kvm-stealtime: cpu 174, msr 17a7cd240
[   22.954970] kvm-clock: cpu 175, msr 1:7ff89bc1, secondary cpu clock
[   23.036439] KVM setup async PF for cpu 175
[   23.063438] kvm-clock: cpu 176, msr 1:7ff89c01, secondary cpu clock
 #176[   23.360889] kvm-stealtime: cpu 175, msr 17a7ed240
[   23.434778] KVM setup async PF for cpu 176
[   23.455974] kvm-clock: cpu 177, msr 1:7ff89c41, secondary cpu clock
 #177[   23.599961] kvm-stealtime: cpu 176, msr 17a80d240
[   23.658365] KVM setup async PF for cpu 177
[   23.681977] kvm-clock: cpu 178, msr 1:7ff89c81, secondary cpu clock
 #178[   23.769905] kvm-stealtime: cpu 177, msr 17a82d240
[   23.838143] KVM setup async PF for cpu 178
[   23.861392] kvm-clock: cpu 179, msr 1:7ff89cc1, secondary cpu clock
 #179[   23.940068] kvm-stealtime: cpu 178, msr 17a84d240
[   23.996590] KVM setup async PF for cpu 179
[   24.014188] kvm-clock: cpu 180, msr 1:7ff89d01, secondary cpu clock
 #180[   24.122088] kvm-stealtime: cpu 179, msr 17a86d240
[   24.176605] KVM setup async PF for cpu 180
[   24.200349] kvm-clock: cpu 181, msr 1:7ff89d41, secondary cpu clock
 #181[   24.309311] kvm-stealtime: cpu 180, msr 17a88d240
[   24.359427] KVM setup async PF for cpu 181
[   24.375437] kvm-clock: cpu 182, msr 1:7ff89d81, secondary cpu clock
 #182[   24.495798] kvm-stealtime: cpu 181, msr 17a8ad240
[   24.553643] KVM setup async PF for cpu 182
[   24.579882] kvm-clock: cpu 183, msr 1:7ff89dc1, secondary cpu clock
 #183[   24.684170] kvm-stealtime: cpu 182, msr 17a8cd240
[   24.739433] KVM setup async PF for cpu 183
[   24.771313] kvm-clock: cpu 184, msr 1:7ff89e01, secondary cpu clock
 #184[   24.853817] kvm-stealtime: cpu 183, msr 17a8ed240
[   24.934593] KVM setup async PF for cpu 184
[   24.952150] kvm-clock: cpu 185, msr 1:7ff89e41, secondary cpu clock
 #185[   25.049139] kvm-stealtime: cpu 184, msr 17a90d240
[   25.109657] KVM setup async PF for cpu 185
[   25.131009] kvm-clock: cpu 186, msr 1:7ff89e81, secondary cpu clock
 #186[   25.227861] kvm-stealtime: cpu 185, msr 17a92d240
[   25.614671] KVM setup async PF for cpu 186
[   25.630524] kvm-clock: cpu 187, msr 1:7ff89ec1, secondary cpu clock
 #187[   25.781934] kvm-stealtime: cpu 186, msr 17a94d240
[   25.832260] KVM setup async PF for cpu 187
[   25.859153] kvm-clock: cpu 188, msr 1:7ff89f01, secondary cpu clock
 #188[   25.980568] kvm-stealtime: cpu 187, msr 17a96d240
 #189[   26.048724] KVM setup async PF for cpu 188
[   26.048726] kvm-stealtime: cpu 188, msr 17a98d240
[   26.173813] kvm-clock: cpu 189, msr 1:7ff89f41, secondary cpu clock
[   26.445695] KVM setup async PF for cpu 189
[   26.471428] kvm-clock: cpu 190, msr 1:7ff89f81, secondary cpu clock
 #190[   26.609815] kvm-stealtime: cpu 189, msr 17a9ad240
[   26.692081] KVM setup async PF for cpu 190
[   26.712631] kvm-clock: cpu 191, msr 1:7ff89fc1, secondary cpu clock
 #191[   26.902914] kvm-stealtime: cpu 190, msr 17a9cd240
[   26.961068] KVM setup async PF for cpu 191
[   26.977596] kvm-clock: cpu 192, msr 1:7ff8a001, secondary cpu clock
 #192[   27.295315] kvm-stealtime: cpu 191, msr 17a9ed240
 #193[   27.366573] KVM setup async PF for cpu 192
[   27.366616] kvm-stealtime: cpu 192, msr 17aa0d240
[   27.476557] kvm-clock: cpu 193, msr 1:7ff8a041, secondary cpu clock
[   27.568238] KVM setup async PF for cpu 193
[   27.587704] kvm-clock: cpu 194, msr 1:7ff8a081, secondary cpu clock
 #194[   27.722552] kvm-stealtime: cpu 193, msr 17aa2d240
[   27.790300] KVM setup async PF for cpu 194
[   27.819743] kvm-clock: cpu 195, msr 1:7ff8a0c1, secondary cpu clock
 #195[   27.924354] kvm-stealtime: cpu 194, msr 17aa4d240
[   27.999097] KVM setup async PF for cpu 195
[   28.023189] kvm-clock: cpu 196, msr 1:7ff8a101, secondary cpu clock
 #196[   28.148422] kvm-stealtime: cpu 195, msr 17aa6d240
[   28.209737] KVM setup async PF for cpu 196
[   28.231625] kvm-clock: cpu 197, msr 1:7ff8a141, secondary cpu clock
 #197[   28.358622] kvm-stealtime: cpu 196, msr 17aa8d240
 #198[   28.693755] KVM setup async PF for cpu 197
[   28.693758] kvm-stealtime: cpu 197, msr 17aaad240
[   28.808949] kvm-clock: cpu 198, msr 1:7ff8a181, secondary cpu clock
[   28.885490] KVM setup async PF for cpu 198
[   28.904436] kvm-clock: cpu 199, msr 1:7ff8a1c1, secondary cpu clock
 #199[   29.035562] kvm-stealtime: cpu 198, msr 17aacd240
[   29.103279] KVM setup async PF for cpu 199
[   29.115891] kvm-clock: cpu 200, msr 1:7ff8a201, secondary cpu clock
 #200[   29.255083] kvm-stealtime: cpu 199, msr 17aaed240
[   29.318143] KVM setup async PF for cpu 200
[   29.337921] kvm-clock: cpu 201, msr 1:7ff8a241, secondary cpu clock
 #201[   29.457278] kvm-stealtime: cpu 200, msr 17ab0d240
[   29.521822] KVM setup async PF for cpu 201
[   29.537974] kvm-clock: cpu 202, msr 1:7ff8a281, secondary cpu clock
 #202[   29.654920] kvm-stealtime: cpu 201, msr 17ab2d240
[   29.859373] KVM setup async PF for cpu 202
[   29.878102] kvm-clock: cpu 203, msr 1:7ff8a2c1, secondary cpu clock
 #203[   30.160221] kvm-stealtime: cpu 202, msr 17ab4d240
[   30.242993] KVM setup async PF for cpu 203
[   30.261985] kvm-clock: cpu 204, msr 1:7ff8a301, secondary cpu clock
 #204[   30.390907] kvm-stealtime: cpu 203, msr 17ab6d240
 #205[   30.460630] KVM setup async PF for cpu 204
[   30.460634] kvm-stealtime: cpu 204, msr 17ab8d240
[   30.585388] kvm-clock: cpu 205, msr 1:7ff8a341, secondary cpu clock
[   30.697727] KVM setup async PF for cpu 205
[   30.815071] kvm-clock: cpu 206, msr 1:7ff8a381, secondary cpu clock
 #206[   30.940430] kvm-stealtime: cpu 205, msr 17abad240
[   31.004829] KVM setup async PF for cpu 206
[   31.017699] kvm-clock: cpu 207, msr 1:7ff8a3c1, secondary cpu clock
 #207[   31.154371] kvm-stealtime: cpu 206, msr 17abcd240
 #208[   31.410014] KVM setup async PF for cpu 207
[   31.410017] kvm-stealtime: cpu 207, msr 17abed240
[   31.532459] kvm-clock: cpu 208, msr 1:7ff8a401, secondary cpu clock
[   31.630771] KVM setup async PF for cpu 208
[   31.652632] kvm-clock: cpu 209, msr 1:7ff8a441, secondary cpu clock
 #209[   31.784642] kvm-stealtime: cpu 208, msr 17ac0d240
[   31.854713] KVM setup async PF for cpu 209
[   31.875896] kvm-clock: cpu 210, msr 1:7ff8a481, secondary cpu clock
 #210[   32.017811] kvm-stealtime: cpu 209, msr 17ac2d240
[   32.089735] KVM setup async PF for cpu 210
[   32.112302] kvm-clock: cpu 211, msr 1:7ff8a4c1, secondary cpu clock
 #211[   32.662666] kvm-stealtime: cpu 210, msr 17ac4d240
[   32.725948] KVM setup async PF for cpu 211
[   32.753712] kvm-clock: cpu 212, msr 1:7ff8a501, secondary cpu clock
 #212[   32.869925] kvm-stealtime: cpu 211, msr 17ac6d240
[   32.945664] KVM setup async PF for cpu 212
[   32.968072] kvm-clock: cpu 213, msr 1:7ff8a541, secondary cpu clock
 #213[   33.100809] kvm-stealtime: cpu 212, msr 17ac8d240
[   33.165779] KVM setup async PF for cpu 213
[   33.184367] kvm-clock: cpu 214, msr 1:7ff8a581, secondary cpu clock
 #214[   33.332325] kvm-stealtime: cpu 213, msr 17acad240
[   33.405731] KVM setup async PF for cpu 214
[   33.424528] kvm-clock: cpu 215, msr 1:7ff8a5c1, secondary cpu clock
 #215[   33.559601] kvm-stealtime: cpu 214, msr 17accd240
[   33.628625] KVM setup async PF for cpu 215
[   33.649272] kvm-clock: cpu 216, msr 1:7ff8a601, secondary cpu clock
 #216[   33.783306] kvm-stealtime: cpu 215, msr 17aced240
[   33.862506] KVM setup async PF for cpu 216
[   33.883262] kvm-clock: cpu 217, msr 1:7ff8a641, secondary cpu clock
 #217[   34.007253] kvm-stealtime: cpu 216, msr 17ad0d240
[   34.074752] KVM setup async PF for cpu 217
[   34.099690] kvm-clock: cpu 218, msr 1:7ff8a681, secondary cpu clock
 #218[   34.219377] kvm-stealtime: cpu 217, msr 17ad2d240
[   34.286292] KVM setup async PF for cpu 218
[   34.350601] kvm-clock: cpu 219, msr 1:7ff8a6c1, secondary cpu clock
 #219[   34.486704] kvm-stealtime: cpu 218, msr 17ad4d240
[   34.563792] KVM setup async PF for cpu 219
[   34.591630] kvm-clock: cpu 220, msr 1:7ff8a701, secondary cpu clock
 #220[   34.743151] kvm-stealtime: cpu 219, msr 17ad6d240
[   34.818484] KVM setup async PF for cpu 220
[   34.836476] kvm-clock: cpu 221, msr 1:7ff8a741, secondary cpu clock
 #221[   35.270464] kvm-stealtime: cpu 220, msr 17ad8d240
[   35.382919] KVM setup async PF for cpu 221
[   35.424146] kvm-clock: cpu 222, msr 1:7ff8a781, secondary cpu clock
 #222[   35.561840] kvm-stealtime: cpu 221, msr 17adad240
[   35.636522] KVM setup async PF for cpu 222
[   35.663725] kvm-clock: cpu 223, msr 1:7ff8a7c1, secondary cpu clock
 #223[   35.952546] kvm-stealtime: cpu 222, msr 17adcd240
[   36.039050] KVM setup async PF for cpu 223
[   36.057374] kvm-clock: cpu 224, msr 1:7ff8a801, secondary cpu clock
 #224[   36.244174] kvm-stealtime: cpu 223, msr 17aded240
[   36.331108] KVM setup async PF for cpu 224
[   36.353090] kvm-clock: cpu 225, msr 1:7ff8a841, secondary cpu clock
 #225[   36.501213] kvm-stealtime: cpu 224, msr 17ae0d240
[   36.572642] KVM setup async PF for cpu 225
[   36.588060] kvm-clock: cpu 226, msr 1:7ff8a881, secondary cpu clock
 #226[   37.052226] kvm-stealtime: cpu 225, msr 17ae2d240
[   37.160796] KVM setup async PF for cpu 226
[   37.180152] kvm-clock: cpu 227, msr 1:7ff8a8c1, secondary cpu clock
 #227[   37.341084] kvm-stealtime: cpu 226, msr 17ae4d240
[   37.583710] KVM setup async PF for cpu 227
[   37.604933] kvm-clock: cpu 228, msr 1:7ff8a901, secondary cpu clock
 #228[   37.749475] kvm-stealtime: cpu 227, msr 17ae6d240
 #229[   37.835023] KVM setup async PF for cpu 228
[   37.835026] kvm-stealtime: cpu 228, msr 17ae8d240
[   37.971413] kvm-clock: cpu 229, msr 1:7ff8a941, secondary cpu clock
[   38.081960] KVM setup async PF for cpu 229
[   38.096460] kvm-clock: cpu 230, msr 1:7ff8a981, secondary cpu clock
 #230[   38.296955] kvm-stealtime: cpu 229, msr 17aead240
[   38.397200] KVM setup async PF for cpu 230
[   38.415398] kvm-clock: cpu 231, msr 1:7ff8a9c1, secondary cpu clock
 #231[   38.592857] kvm-stealtime: cpu 230, msr 17aecd240
[   38.684993] KVM setup async PF for cpu 231
[   38.713272] kvm-clock: cpu 232, msr 1:7ff8aa01, secondary cpu clock
 #232[   38.852942] kvm-stealtime: cpu 231, msr 17aeed240
[   38.940768] KVM setup async PF for cpu 232
[   38.960987] kvm-clock: cpu 233, msr 1:7ff8aa41, secondary cpu clock
 #233[   39.092097] kvm-stealtime: cpu 232, msr 17af0d240
[   39.163269] KVM setup async PF for cpu 233
[   39.186362] kvm-clock: cpu 234, msr 1:7ff8aa81, secondary cpu clock
 #234[   39.351775] kvm-stealtime: cpu 233, msr 17af2d240
[   39.601624] KVM setup async PF for cpu 234
[   39.624590] kvm-clock: cpu 235, msr 1:7ff8aac1, secondary cpu clock
 #235[   39.770515] kvm-stealtime: cpu 234, msr 17af4d240
[   39.859798] KVM setup async PF for cpu 235
[   39.883598] kvm-clock: cpu 236, msr 1:7ff8ab01, secondary cpu clock
 #236[   40.062295] kvm-stealtime: cpu 235, msr 17af6d240
[   40.135641] KVM setup async PF for cpu 236
[   40.154040] kvm-clock: cpu 237, msr 1:7ff8ab41, secondary cpu clock
 #237[   40.586912] kvm-stealtime: cpu 236, msr 17af8d240
 #238[   40.682162] KVM setup async PF for cpu 237
[   40.682165] kvm-stealtime: cpu 237, msr 17afad240
[   40.868178] kvm-clock: cpu 238, msr 1:7ff8ab81, secondary cpu clock
[   40.967259] KVM setup async PF for cpu 238
[   41.185602] kvm-clock: cpu 239, msr 1:7ff8abc1, secondary cpu clock
 #239[   41.359526] kvm-stealtime: cpu 238, msr 17afcd240
[   41.443333] KVM setup async PF for cpu 239
[   41.464447] kvm-clock: cpu 240, msr 1:7ff8ac01, secondary cpu clock
 #240[   41.621175] kvm-stealtime: cpu 239, msr 17afed240
[   41.942080] KVM setup async PF for cpu 240
[   41.967161] kvm-clock: cpu 241, msr 1:7ff8ac41, secondary cpu clock
 #241[   42.152256] kvm-stealtime: cpu 240, msr 17b00d240
 #242[   42.225931] KVM setup async PF for cpu 241
[   42.245474] kvm-clock: cpu 242, msr 1:7ff8ac81, secondary cpu clock
[   42.422471] kvm-stealtime: cpu 241, msr 17b02d240
[   42.510088] KVM setup async PF for cpu 242
[   42.545386] kvm-clock: cpu 243, msr 1:7ff8acc1, secondary cpu clock
 #243[   43.275985] kvm-stealtime: cpu 242, msr 17b04d240
 #244[   43.412311] KVM setup async PF for cpu 243
[   43.412314] kvm-stealtime: cpu 243, msr 17b06d240
[   43.574749] kvm-clock: cpu 244, msr 1:7ff8ad01, secondary cpu clock
[   43.688679] KVM setup async PF for cpu 244
[   43.732921] kvm-clock: cpu 245, msr 1:7ff8ad41, secondary cpu clock
 #245[   43.872098] kvm-stealtime: cpu 244, msr 17b08d240
[   43.948700] KVM setup async PF for cpu 245
[   43.966433] kvm-clock: cpu 246, msr 1:7ff8ad81, secondary cpu clock
 #246[   44.129948] kvm-stealtime: cpu 245, msr 17b0ad240
[   44.211083] KVM setup async PF for cpu 246
[   44.244217] kvm-clock: cpu 247, msr 1:7ff8adc1, secondary cpu clock
 #247[   44.415566] kvm-stealtime: cpu 246, msr 17b0cd240
[   44.498987] KVM setup async PF for cpu 247
[   44.514635] kvm-clock: cpu 248, msr 1:7ff8ae01, secondary cpu clock
 #248[   44.688137] kvm-stealtime: cpu 247, msr 17b0ed240
[   44.919950] KVM setup async PF for cpu 248
[   44.948259] kvm-clock: cpu 249, msr 1:7ff8ae41, secondary cpu clock
 #249[   45.249429] kvm-stealtime: cpu 248, msr 17b10d240
[   45.331717] KVM setup async PF for cpu 249
[   45.353609] kvm-clock: cpu 250, msr 1:7ff8ae81, secondary cpu clock
 #250[   45.533646] kvm-stealtime: cpu 249, msr 17b12d240
[   45.619286] KVM setup async PF for cpu 250
[   45.642058] kvm-clock: cpu 251, msr 1:7ff8aec1, secondary cpu clock
 #251[   45.973760] kvm-stealtime: cpu 250, msr 17b14d240
[   46.052959] KVM setup async PF for cpu 251
[   46.087349] kvm-clock: cpu 252, msr 1:7ff8af01, secondary cpu clock
 #252[   46.251748] kvm-stealtime: cpu 251, msr 17b16d240
[   46.340756] KVM setup async PF for cpu 252
[   46.369129] kvm-clock: cpu 253, msr 1:7ff8af41, secondary cpu clock
 #253[   46.511055] kvm-stealtime: cpu 252, msr 17b18d240
 #254[   46.615572] KVM setup async PF for cpu 253
[   46.615575] kvm-stealtime: cpu 253, msr 17b1ad240
[   46.788464] kvm-clock: cpu 254, msr 1:7ff8af81, secondary cpu clock
[   46.907756] KVM setup async PF for cpu 254
[   46.947097] kvm-clock: cpu 255, msr 1:7ff8afc1, secondary cpu clock
 #255[   47.187018] kvm-stealtime: cpu 254, msr 17b1cd240
[   47.280675] KVM setup async PF for cpu 255
[   47.302530] kvm-clock: cpu 256, msr 1:7ff8b001, secondary cpu clock
 #256[   47.480134] kvm-stealtime: cpu 255, msr 17b1ed240
[   47.569897] KVM setup async PF for cpu 256
[   47.592042] kvm-clock: cpu 257, msr 1:7ff8b041, secondary cpu clock
 #257[   47.739077] kvm-stealtime: cpu 256, msr 17b20d240
[   47.831786] KVM setup async PF for cpu 257
[   47.863761] kvm-clock: cpu 258, msr 1:7ff8b081, secondary cpu clock
 #258[   48.027584] kvm-stealtime: cpu 257, msr 17b22d240
 #259[   48.423805] KVM setup async PF for cpu 258
[   48.423808] kvm-stealtime: cpu 258, msr 17b24d240
[   48.568149] kvm-clock: cpu 259, msr 1:7ff8b0c1, secondary cpu clock
[   48.678071] KVM setup async PF for cpu 259
[   48.706616] kvm-clock: cpu 260, msr 1:7ff8b101, secondary cpu clock
 #260[   48.933192] kvm-stealtime: cpu 259, msr 17b26d240
[   49.194606] KVM setup async PF for cpu 260
[   49.211844] kvm-clock: cpu 261, msr 1:7ff8b141, secondary cpu clock
 #261[   50.231731] kvm-stealtime: cpu 260, msr 17b28d240
[   50.338947] KVM setup async PF for cpu 261
[   50.380883] kvm-clock: cpu 262, msr 1:7ff8b181, secondary cpu clock
 #262[   50.524290] kvm-stealtime: cpu 261, msr 17b2ad240
[   50.605868] KVM setup async PF for cpu 262
[   50.633146] kvm-clock: cpu 263, msr 1:7ff8b1c1, secondary cpu clock
 #263[   50.796874] kvm-stealtime: cpu 262, msr 17b2cd240
[   50.872994] KVM setup async PF for cpu 263
[   50.900886] kvm-clock: cpu 264, msr 1:7ff8b201, secondary cpu clock
 #264[   51.078723] kvm-stealtime: cpu 263, msr 17b2ed240
[   51.165197] KVM setup async PF for cpu 264
[   51.185338] kvm-clock: cpu 265, msr 1:7ff8b241, secondary cpu clock
 #265[   51.359198] kvm-stealtime: cpu 264, msr 17b30d240
[   51.871406] KVM setup async PF for cpu 265
[   51.900882] kvm-clock: cpu 266, msr 1:7ff8b281, secondary cpu clock
 #266[   52.147902] kvm-stealtime: cpu 265, msr 17b32d240
 #267[   52.290053] KVM setup async PF for cpu 266
[   52.290056] kvm-stealtime: cpu 266, msr 17b34d240
[   52.460120] kvm-clock: cpu 267, msr 1:7ff8b2c1, secondary cpu clock
[   52.578927] KVM setup async PF for cpu 267
[   52.604610] kvm-clock: cpu 268, msr 1:7ff8b301, secondary cpu clock
 #268[   52.785099] kvm-stealtime: cpu 267, msr 17b36d240
[   53.028564] KVM setup async PF for cpu 268
[   53.048554] kvm-clock: cpu 269, msr 1:7ff8b341, secondary cpu clock
 #269[   53.213391] kvm-stealtime: cpu 268, msr 17b38d240
[   53.321327] KVM setup async PF for cpu 269
[   53.355509] kvm-clock: cpu 270, msr 1:7ff8b381, secondary cpu clock
 #270[   53.541269] kvm-stealtime: cpu 269, msr 17b3ad240
[   53.885865] KVM setup async PF for cpu 270
[   53.904477] kvm-clock: cpu 271, msr 1:7ff8b3c1, secondary cpu clock
 #271[   54.092509] kvm-stealtime: cpu 270, msr 17b3cd240
[   54.310303] KVM setup async PF for cpu 271
[   54.339445] kvm-clock: cpu 272, msr 1:7ff8b401, secondary cpu clock
 #272[   55.001267] kvm-stealtime: cpu 271, msr 17b3ed240
[   55.435805] KVM setup async PF for cpu 272
[   55.458530] kvm-clock: cpu 273, msr 1:7ff8b441, secondary cpu clock
 #273[   55.698996] kvm-stealtime: cpu 272, msr 17b40d240
[   55.826903] KVM setup async PF for cpu 273
[   55.862419] kvm-clock: cpu 274, msr 1:7ff8b481, secondary cpu clock
 #274[   56.051174] kvm-stealtime: cpu 273, msr 17b42d240
[   56.141593] KVM setup async PF for cpu 274
[   56.174281] kvm-clock: cpu 275, msr 1:7ff8b4c1, secondary cpu clock
 #275[   56.346306] kvm-stealtime: cpu 274, msr 17b44d240
[   56.446713] KVM setup async PF for cpu 275
[   56.477538] kvm-clock: cpu 276, msr 1:7ff8b501, secondary cpu clock
 #276[   56.995183] kvm-stealtime: cpu 275, msr 17b46d240
[   57.179603] KVM setup async PF for cpu 276
[   57.206930] kvm-clock: cpu 277, msr 1:7ff8b541, secondary cpu clock
 #277[   57.398031] kvm-stealtime: cpu 276, msr 17b48d240
[   57.502385] KVM setup async PF for cpu 277
[   57.521610] kvm-clock: cpu 278, msr 1:7ff8b581, secondary cpu clock
 #278[   57.742213] kvm-stealtime: cpu 277, msr 17b4ad240
[   57.842249] KVM setup async PF for cpu 278
[   57.866035] kvm-clock: cpu 279, msr 1:7ff8b5c1, secondary cpu clock
 #279[   58.047672] kvm-stealtime: cpu 278, msr 17b4cd240
[   58.282019] KVM setup async PF for cpu 279
[   58.315199] kvm-clock: cpu 280, msr 1:7ff8b601, secondary cpu clock
 #280[   58.502495] kvm-stealtime: cpu 279, msr 17b4ed240
[   59.175311] KVM setup async PF for cpu 280
[   59.205849] kvm-clock: cpu 281, msr 1:7ff8b641, secondary cpu clock
 #281[   59.401340] kvm-stealtime: cpu 280, msr 17b50d240
[   60.019877] KVM setup async PF for cpu 281
[   60.047461] kvm-clock: cpu 282, msr 1:7ff8b681, secondary cpu clock
 #282[   60.258420] kvm-stealtime: cpu 281, msr 17b52d240
 #283[   60.362500] KVM setup async PF for cpu 282
[   60.362504] kvm-stealtime: cpu 282, msr 17b54d240
[   60.531702] kvm-clock: cpu 283, msr 1:7ff8b6c1, secondary cpu clock
[   60.671090] KVM setup async PF for cpu 283
[   60.708282] kvm-clock: cpu 284, msr 1:7ff8b701, secondary cpu clock
 #284[   61.163256] kvm-stealtime: cpu 283, msr 17b56d240
[   61.258812] KVM setup async PF for cpu 284
[   61.285684] kvm-clock: cpu 285, msr 1:7ff8b741, secondary cpu clock
 #285[   61.447688] kvm-stealtime: cpu 284, msr 17b58d240
[   61.533383] KVM setup async PF for cpu 285
[   61.573242] kvm-clock: cpu 286, msr 1:7ff8b781, secondary cpu clock
 #286[   61.744518] kvm-stealtime: cpu 285, msr 17b5ad240
[   61.863337] KVM setup async PF for cpu 286
[   61.886111] kvm-clock: cpu 287, msr 1:7ff8b7c1, secondary cpu clock
 #287 OK
[   62.144795] kvm-stealtime: cpu 286, msr 17b5cd240
[   62.272568] Brought up 288 CPUs
[   62.272587] KVM setup async PF for cpu 287
[   62.272593] kvm-stealtime: cpu 287, msr 17b5ed240
[   62.537632] smpboot: Total of 288 processors activated (2068808.83 BogoMIPS)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[   63.266889] devtmpfs: initialized
[   63.331827] EVM: security.selinux
[   63.396014] EVM: security.ima
[   63.449920] EVM: security.capability
[   63.559089] atomic64 test passed for x86-64 platform with CX8 and with SSE
[   63.678783] NET: Registered protocol family 16
[   64.913931] ACPI: bus type PCI registered
[   64.983827] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[   65.112679] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xb0000000-0xbfffffff] (base 0xb0000000)
[   65.270757] PCI: MMCONFIG at [mem 0xb0000000-0xbfffffff] reserved in E820
[   65.391425] PCI: Using configuration type 1 for base access
[   65.509962] ACPI: Added _OSI(Module Device)
[   65.605014] ACPI: Added _OSI(Processor Device)
[   65.687318] ACPI: Added _OSI(3.0 _SCP Extensions)
[   65.776012] ACPI: Added _OSI(Processor Aggregator Device)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 9
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 57, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[   65.889560] ACPI: Interpreter enabled
[   65.966884] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20130517/hwxface-571)
[   66.157520] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20130517/hwxface-571)
[   66.316657] ACPI: (supports S0 S3 S4 S5)
[   66.395711] ACPI: Using IOAPIC for interrupt routing
[   66.518386] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[   66.714142] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[   66.857066] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[   67.025858] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability]
[   67.178947] PCI host bridge to bus 0000:00
[   67.262027] pci_bus 0000:00: root bus resource [bus 00-ff]
[   67.386760] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
[   67.607800] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
[   67.725481] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[   67.850435] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff]
[   68.040779] pci 0000:00:1f.0: quirk: [io  0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO
[   68.238729] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[   68.888212] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[   68.996318] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[   69.110333] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[   69.591253] ACPI: PCI Interrupt Link [LNKE] (IRQs 5 *10 11)
[   69.686937] ACPI: PCI Interrupt Link [LNKF] (IRQs 5 *10 11)
[   69.791437] ACPI: PCI Interrupt Link [LNKG] (IRQs 5 10 *11)
[   69.933268] ACPI: PCI Interrupt Link [LNKH] (IRQs 5 10 *11)
[   70.294510] ACPI: PCI Interrupt Link [GSIA] (IRQs *16)
[   70.392932] ACPI: PCI Interrupt Link [GSIB] (IRQs *17)
[   70.507695] ACPI: PCI Interrupt Link [GSIC] (IRQs *18)
[   70.626821] ACPI: PCI Interrupt Link [GSID] (IRQs *19)
[   70.730802] ACPI: PCI Interrupt Link [GSIE] (IRQs *20)
[   70.846990] ACPI: PCI Interrupt Link [GSIF] (IRQs *21)
[   70.956455] ACPI: PCI Interrupt Link [GSIG] (IRQs *22)
[   71.070072] ACPI: PCI Interrupt Link [GSIH] (IRQs *23)
[   71.194772] ACPI: Enabled 2 GPEs in block 00 to 3F
[   71.294325] vgaarb: device added: PCI:0000:00:01.0,decodes=io+mem,owns=io+mem,locks=none
[   72.075843] vgaarb: loaded
[   72.119446] vgaarb: bridge control possible 0000:00:01.0
[   72.220541] SCSI subsystem initialized
[   72.277274] ACPI: bus type USB registered
[   72.364390] usbcore: registered new interface driver usbfs
[   72.478120] usbcore: registered new interface driver hub
[   72.619428] usbcore: registered new device driver usb
[   72.749525] PCI: Using ACPI for IRQ routing
[   72.856388] NetLabel: Initializing
[   72.916624] NetLabel:  domain hash size = 128
[   73.014210] NetLabel:  protocols = UNLABELED CIPSOv4
[   73.109293] NetLabel:  unlabeled traffic allowed by default
[   73.254755] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[   73.409014] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[   73.552137] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[   74.201738] Switching to clocksource kvm-clock
[   74.577349] pnp: PnP ACPI init
[   74.612830] ACPI: bus type PNP registered
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 8
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 7
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 4
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[   74.665520] pnp: PnP ACPI: found 6 devices
[   74.714367] ACPI: bus type PNP unregistered
[   74.764761] NET: Registered protocol family 2
[   74.816472] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[   74.895310] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[   74.965732] TCP: Hash tables configured (established 32768 bind 32768)
[   75.034251] TCP: reno registered
[   75.070803] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[   75.135618] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[   75.201739] NET: Registered protocol family 1
[   75.249114] Unpacking initramfs...
[   75.461243] Freeing initrd memory: 19272k freed
[   75.517440] DMAR: No RMRR found
[   75.556391] DMAR: No ATSR found
[   75.588459] IOMMU 0 0xfed90000: using Queued invalidation
[   75.637069] IOMMU: Setting RMRR:
[   75.674019] IOMMU: Prepare 0-16MiB unity mapping for LPC
[   75.714625] IOMMU: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
(vtd)vtd_mem_write: DMAR_RTADDR_REG write addr 0x20, size 4, val 0x6e7cd000
(vtd)vtd_mem_write: DMAR_RTADDR_REG_HI write addr 0x24, size 4, val 0x1
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x47000000
(vtd)vtd_handle_gcmd_write: value 0x47000000 status 0x7000000
(vtd)vtd_handle_gcmd_srtp: set Root Table Pointer
(vtd)vtd_root_table_setup: root_table addr 0x16e7cd000 
(vtd)vtd_handle_gcmd_sirtp: set Interrupt Remap Table Pointer
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_table_setup: int remap table addr 0x177100000 size 65536
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x47000000
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x87000000
(vtd)vtd_handle_gcmd_write: value 0x87000000 status 0x47000000
(vtd)vtd_handle_gcmd_te: Translation Enable on
(vtd)vtd_handle_gcmd_sirtp: set Interrupt Remap Table Pointer
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_table_setup: int remap table addr 0x177100000 size 65536
(vtd)vtd_mem_read: addr 0x1c size 4 val 0xc7000000
(vtd)vtd_mem_read: addr 0x64 size 4 val 0x0
(vtd)vtd_mem_write: DMAR_PMEN_REG write addr 0x64, size 4, val 0x0
(vtd)vtd_mem_read: addr 0x64 size 4 val 0x0
[   75.788347] PCI-DMA: Intel(R) Virtualization Technology for Directed I/O
[   75.871126] microcode: CPU0 sig=0x663, pf=0x1, revision=0x1
[   75.922692] microcode: CPU1 sig=0x663, pf=0x1, revision=0x1
[   75.974396] microcode: CPU2 sig=0x663, pf=0x1, revision=0x1
[   76.030804] microcode: CPU3 sig=0x663, pf=0x1, revision=0x1
[   76.098151] microcode: CPU4 sig=0x663, pf=0x1, revision=0x1
[   76.157000] microcode: CPU5 sig=0x663, pf=0x1, revision=0x1
[   76.214773] microcode: CPU6 sig=0x663, pf=0x1, revision=0x1
[   76.275197] microcode: CPU7 sig=0x663, pf=0x1, revision=0x1
[   76.325243] microcode: CPU8 sig=0x663, pf=0x1, revision=0x1
[   76.372926] microcode: CPU9 sig=0x663, pf=0x1, revision=0x1
[   76.432276] microcode: CPU10 sig=0x663, pf=0x1, revision=0x1
[   76.484242] microcode: CPU11 sig=0x663, pf=0x1, revision=0x1
[   76.540526] microcode: CPU12 sig=0x663, pf=0x1, revision=0x1
[   76.603581] microcode: CPU13 sig=0x663, pf=0x1, revision=0x1
[   76.647596] microcode: CPU14 sig=0x663, pf=0x1, revision=0x1
[   76.707560] microcode: CPU15 sig=0x663, pf=0x1, revision=0x1
[   76.768785] microcode: CPU16 sig=0x663, pf=0x1, revision=0x1
[   76.824918] microcode: CPU17 sig=0x663, pf=0x1, revision=0x1
[   76.877617] tsc: Refined TSC clocksource calibration: 3591.680 MHz
[   76.943040] microcode: CPU18 sig=0x663, pf=0x1, revision=0x1
[   77.001675] microcode: CPU19 sig=0x663, pf=0x1, revision=0x1
[   77.062462] microcode: CPU20 sig=0x663, pf=0x1, revision=0x1
[   77.119534] microcode: CPU21 sig=0x663, pf=0x1, revision=0x1
[   77.175357] microcode: CPU22 sig=0x663, pf=0x1, revision=0x1
[   77.226283] microcode: CPU23 sig=0x663, pf=0x1, revision=0x1
[   77.288748] microcode: CPU24 sig=0x663, pf=0x1, revision=0x1
[   77.348893] microcode: CPU25 sig=0x663, pf=0x1, revision=0x1
[   77.393587] microcode: CPU26 sig=0x663, pf=0x1, revision=0x1
[   77.444283] microcode: CPU27 sig=0x663, pf=0x1, revision=0x1
[   77.493185] microcode: CPU28 sig=0x663, pf=0x1, revision=0x1
[   77.544734] microcode: CPU29 sig=0x663, pf=0x1, revision=0x1
[   77.608301] microcode: CPU30 sig=0x663, pf=0x1, revision=0x1
[   77.667990] microcode: CPU31 sig=0x663, pf=0x1, revision=0x1
[   77.723623] microcode: CPU32 sig=0x663, pf=0x1, revision=0x1
[   77.783024] microcode: CPU33 sig=0x663, pf=0x1, revision=0x1
[   77.839749] microcode: CPU34 sig=0x663, pf=0x1, revision=0x1
[   77.905081] microcode: CPU35 sig=0x663, pf=0x1, revision=0x1
[   77.975976] microcode: CPU36 sig=0x663, pf=0x1, revision=0x1
[   78.038600] microcode: CPU37 sig=0x663, pf=0x1, revision=0x1
[   78.105345] microcode: CPU38 sig=0x663, pf=0x1, revision=0x1
[   78.169743] microcode: CPU39 sig=0x663, pf=0x1, revision=0x1
[   78.224829] microcode: CPU40 sig=0x663, pf=0x1, revision=0x1
[   78.276146] microcode: CPU41 sig=0x663, pf=0x1, revision=0x1
[   78.330293] microcode: CPU42 sig=0x663, pf=0x1, revision=0x1
[   78.391639] microcode: CPU43 sig=0x663, pf=0x1, revision=0x1
[   78.449676] microcode: CPU44 sig=0x663, pf=0x1, revision=0x1
[   78.499222] microcode: CPU45 sig=0x663, pf=0x1, revision=0x1
[   78.555913] microcode: CPU46 sig=0x663, pf=0x1, revision=0x1
[   78.611884] microcode: CPU47 sig=0x663, pf=0x1, revision=0x1
[   78.678300] microcode: CPU48 sig=0x663, pf=0x1, revision=0x1
[   78.734380] microcode: CPU49 sig=0x663, pf=0x1, revision=0x1
[   78.792042] microcode: CPU50 sig=0x663, pf=0x1, revision=0x1
[   78.854864] microcode: CPU51 sig=0x663, pf=0x1, revision=0x1
[   78.908604] microcode: CPU52 sig=0x663, pf=0x1, revision=0x1
[   78.967648] microcode: CPU53 sig=0x663, pf=0x1, revision=0x1
[   79.025677] microcode: CPU54 sig=0x663, pf=0x1, revision=0x1
[   79.076828] microcode: CPU55 sig=0x663, pf=0x1, revision=0x1
[   79.136643] microcode: CPU56 sig=0x663, pf=0x1, revision=0x1
[   79.200313] microcode: CPU57 sig=0x663, pf=0x1, revision=0x1
[   79.249720] microcode: CPU58 sig=0x663, pf=0x1, revision=0x1
[   79.308324] microcode: CPU59 sig=0x663, pf=0x1, revision=0x1
[   79.362208] microcode: CPU60 sig=0x663, pf=0x1, revision=0x1
[   79.408753] microcode: CPU61 sig=0x663, pf=0x1, revision=0x1
[   79.468814] microcode: CPU62 sig=0x663, pf=0x1, revision=0x1
[   79.526450] microcode: CPU63 sig=0x663, pf=0x1, revision=0x1
[   79.584283] microcode: CPU64 sig=0x663, pf=0x1, revision=0x1
[   79.636940] microcode: CPU65 sig=0x663, pf=0x1, revision=0x1
[   79.694135] microcode: CPU66 sig=0x663, pf=0x1, revision=0x1
[   79.757871] microcode: CPU67 sig=0x663, pf=0x1, revision=0x1
[   79.813836] microcode: CPU68 sig=0x663, pf=0x1, revision=0x1
[   79.868335] microcode: CPU69 sig=0x663, pf=0x1, revision=0x1
[   79.926294] microcode: CPU70 sig=0x663, pf=0x1, revision=0x1
[   79.981544] microcode: CPU71 sig=0x663, pf=0x1, revision=0x1
[   80.046337] microcode: CPU72 sig=0x663, pf=0x1, revision=0x1
[   80.116789] microcode: CPU73 sig=0x663, pf=0x1, revision=0x1
[   80.189686] microcode: CPU74 sig=0x663, pf=0x1, revision=0x1
[   80.250035] microcode: CPU75 sig=0x663, pf=0x1, revision=0x1
[   80.318871] microcode: CPU76 sig=0x663, pf=0x1, revision=0x1
[   80.387216] microcode: CPU77 sig=0x663, pf=0x1, revision=0x1
[   80.449368] microcode: CPU78 sig=0x663, pf=0x1, revision=0x1
[   80.512204] microcode: CPU79 sig=0x663, pf=0x1, revision=0x1
[   80.570012] microcode: CPU80 sig=0x663, pf=0x1, revision=0x1
[   80.631407] microcode: CPU81 sig=0x663, pf=0x1, revision=0x1
[   80.692441] microcode: CPU82 sig=0x663, pf=0x1, revision=0x1
[   80.754529] microcode: CPU83 sig=0x663, pf=0x1, revision=0x1
[   80.821016] microcode: CPU84 sig=0x663, pf=0x1, revision=0x1
[   80.873537] microcode: CPU85 sig=0x663, pf=0x1, revision=0x1
[   80.935737] microcode: CPU86 sig=0x663, pf=0x1, revision=0x1
[   80.991603] microcode: CPU87 sig=0x663, pf=0x1, revision=0x1
[   81.057505] microcode: CPU88 sig=0x663, pf=0x1, revision=0x1
[   81.115224] microcode: CPU89 sig=0x663, pf=0x1, revision=0x1
[   81.176704] microcode: CPU90 sig=0x663, pf=0x1, revision=0x1
[   81.239586] microcode: CPU91 sig=0x663, pf=0x1, revision=0x1
[   81.302278] microcode: CPU92 sig=0x663, pf=0x1, revision=0x1
[   81.355916] microcode: CPU93 sig=0x663, pf=0x1, revision=0x1
[   81.425745] microcode: CPU94 sig=0x663, pf=0x1, revision=0x1
[   81.489129] microcode: CPU95 sig=0x663, pf=0x1, revision=0x1
[   81.554009] microcode: CPU96 sig=0x663, pf=0x1, revision=0x1
[   81.610826] microcode: CPU97 sig=0x663, pf=0x1, revision=0x1
[   81.666423] microcode: CPU98 sig=0x663, pf=0x1, revision=0x1
[   81.721463] microcode: CPU99 sig=0x663, pf=0x1, revision=0x1
[   81.770470] microcode: CPU100 sig=0x663, pf=0x1, revision=0x1
[   81.822027] microcode: CPU101 sig=0x663, pf=0x1, revision=0x1
[   81.879519] microcode: CPU102 sig=0x663, pf=0x1, revision=0x1
[   81.936365] microcode: CPU103 sig=0x663, pf=0x1, revision=0x1
[   81.999251] microcode: CPU104 sig=0x663, pf=0x1, revision=0x1
[   82.067215] microcode: CPU105 sig=0x663, pf=0x1, revision=0x1
[   82.123555] microcode: CPU106 sig=0x663, pf=0x1, revision=0x1
[   82.164402] microcode: CPU107 sig=0x663, pf=0x1, revision=0x1
[   82.221518] microcode: CPU108 sig=0x663, pf=0x1, revision=0x1
[   82.277393] microcode: CPU109 sig=0x663, pf=0x1, revision=0x1
[   82.338223] microcode: CPU110 sig=0x663, pf=0x1, revision=0x1
[   82.394127] microcode: CPU111 sig=0x663, pf=0x1, revision=0x1
[   82.447930] microcode: CPU112 sig=0x663, pf=0x1, revision=0x1
[   82.499263] microcode: CPU113 sig=0x663, pf=0x1, revision=0x1
[   82.549061] microcode: CPU114 sig=0x663, pf=0x1, revision=0x1
[   82.601072] microcode: CPU115 sig=0x663, pf=0x1, revision=0x1
[   82.662965] microcode: CPU116 sig=0x663, pf=0x1, revision=0x1
[   82.717451] microcode: CPU117 sig=0x663, pf=0x1, revision=0x1
[   82.766506] microcode: CPU118 sig=0x663, pf=0x1, revision=0x1
[   82.827389] microcode: CPU119 sig=0x663, pf=0x1, revision=0x1
[   82.894035] microcode: CPU120 sig=0x663, pf=0x1, revision=0x1
[   82.953140] microcode: CPU121 sig=0x663, pf=0x1, revision=0x1
[   83.015285] microcode: CPU122 sig=0x663, pf=0x1, revision=0x1
[   83.080302] microcode: CPU123 sig=0x663, pf=0x1, revision=0x1
[   83.143453] microcode: CPU124 sig=0x663, pf=0x1, revision=0x1
[   83.226843] microcode: CPU125 sig=0x663, pf=0x1, revision=0x1
[   83.279160] microcode: CPU126 sig=0x663, pf=0x1, revision=0x1
[   83.337908] microcode: CPU127 sig=0x663, pf=0x1, revision=0x1
[   83.399834] microcode: CPU128 sig=0x663, pf=0x1, revision=0x1
[   83.462551] microcode: CPU129 sig=0x663, pf=0x1, revision=0x1
[   83.521585] microcode: CPU130 sig=0x663, pf=0x1, revision=0x1
[   83.585840] microcode: CPU131 sig=0x663, pf=0x1, revision=0x1
[   83.643400] microcode: CPU132 sig=0x663, pf=0x1, revision=0x1
[   83.695910] microcode: CPU133 sig=0x663, pf=0x1, revision=0x1
[   83.759238] microcode: CPU134 sig=0x663, pf=0x1, revision=0x1
[   83.825375] microcode: CPU135 sig=0x663, pf=0x1, revision=0x1
[   83.889462] microcode: CPU136 sig=0x663, pf=0x1, revision=0x1
[   83.946221] microcode: CPU137 sig=0x663, pf=0x1, revision=0x1
[   84.006507] microcode: CPU138 sig=0x663, pf=0x1, revision=0x1
[   84.064980] microcode: CPU139 sig=0x663, pf=0x1, revision=0x1
[   84.127391] microcode: CPU140 sig=0x663, pf=0x1, revision=0x1
[   84.193647] microcode: CPU141 sig=0x663, pf=0x1, revision=0x1
[   84.265561] microcode: CPU142 sig=0x663, pf=0x1, revision=0x1
[   84.320533] microcode: CPU143 sig=0x663, pf=0x1, revision=0x1
[   84.370602] microcode: CPU144 sig=0x663, pf=0x1, revision=0x1
[   84.430007] microcode: CPU145 sig=0x663, pf=0x1, revision=0x1
[   84.489411] microcode: CPU146 sig=0x663, pf=0x1, revision=0x1
[   84.549319] microcode: CPU147 sig=0x663, pf=0x1, revision=0x1
[   84.606254] microcode: CPU148 sig=0x663, pf=0x1, revision=0x1
[   84.666509] microcode: CPU149 sig=0x663, pf=0x1, revision=0x1
[   84.707813] microcode: CPU150 sig=0x663, pf=0x1, revision=0x1
[   84.765282] microcode: CPU151 sig=0x663, pf=0x1, revision=0x1
[   84.818389] microcode: CPU152 sig=0x663, pf=0x1, revision=0x1
[   84.885371] microcode: CPU153 sig=0x663, pf=0x1, revision=0x1
[   84.938484] microcode: CPU154 sig=0x663, pf=0x1, revision=0x1
[   84.998757] microcode: CPU155 sig=0x663, pf=0x1, revision=0x1
[   85.052748] microcode: CPU156 sig=0x663, pf=0x1, revision=0x1
[   85.111823] microcode: CPU157 sig=0x663, pf=0x1, revision=0x1
[   85.168851] microcode: CPU158 sig=0x663, pf=0x1, revision=0x1
[   85.231380] microcode: CPU159 sig=0x663, pf=0x1, revision=0x1
[   85.292004] microcode: CPU160 sig=0x663, pf=0x1, revision=0x1
[   85.353980] microcode: CPU161 sig=0x663, pf=0x1, revision=0x1
[   85.401715] microcode: CPU162 sig=0x663, pf=0x1, revision=0x1
[   85.456740] microcode: CPU163 sig=0x663, pf=0x1, revision=0x1
[   85.524085] microcode: CPU164 sig=0x663, pf=0x1, revision=0x1
[   85.574655] microcode: CPU165 sig=0x663, pf=0x1, revision=0x1
[   85.631838] microcode: CPU166 sig=0x663, pf=0x1, revision=0x1
[   85.693772] microcode: CPU167 sig=0x663, pf=0x1, revision=0x1
[   85.746557] microcode: CPU168 sig=0x663, pf=0x1, revision=0x1
[   85.798646] microcode: CPU169 sig=0x663, pf=0x1, revision=0x1
[   85.850445] microcode: CPU170 sig=0x663, pf=0x1, revision=0x1
[   85.911216] microcode: CPU171 sig=0x663, pf=0x1, revision=0x1
[   85.970782] microcode: CPU172 sig=0x663, pf=0x1, revision=0x1
[   86.015292] microcode: CPU173 sig=0x663, pf=0x1, revision=0x1
[   86.061962] microcode: CPU174 sig=0x663, pf=0x1, revision=0x1
[   86.106630] microcode: CPU175 sig=0x663, pf=0x1, revision=0x1
[   86.160398] microcode: CPU176 sig=0x663, pf=0x1, revision=0x1
[   86.224971] microcode: CPU177 sig=0x663, pf=0x1, revision=0x1
[   86.281286] microcode: CPU178 sig=0x663, pf=0x1, revision=0x1
[   86.338970] microcode: CPU179 sig=0x663, pf=0x1, revision=0x1
[   86.391209] microcode: CPU180 sig=0x663, pf=0x1, revision=0x1
[   86.440226] microcode: CPU181 sig=0x663, pf=0x1, revision=0x1
[   86.506489] microcode: CPU182 sig=0x663, pf=0x1, revision=0x1
[   86.566006] microcode: CPU183 sig=0x663, pf=0x1, revision=0x1
[   86.627260] microcode: CPU184 sig=0x663, pf=0x1, revision=0x1
[   86.681552] microcode: CPU185 sig=0x663, pf=0x1, revision=0x1
[   86.741038] microcode: CPU186 sig=0x663, pf=0x1, revision=0x1
[   86.798542] microcode: CPU187 sig=0x663, pf=0x1, revision=0x1
[   86.836064] microcode: CPU188 sig=0x663, pf=0x1, revision=0x1
[   86.894488] microcode: CPU189 sig=0x663, pf=0x1, revision=0x1
[   86.953499] microcode: CPU190 sig=0x663, pf=0x1, revision=0x1
[   87.005787] microcode: CPU191 sig=0x663, pf=0x1, revision=0x1
[   87.067771] microcode: CPU192 sig=0x663, pf=0x1, revision=0x1
[   87.119557] microcode: CPU193 sig=0x663, pf=0x1, revision=0x1
[   87.177397] microcode: CPU194 sig=0x663, pf=0x1, revision=0x1
[   87.225340] microcode: CPU195 sig=0x663, pf=0x1, revision=0x1
[   87.288880] microcode: CPU196 sig=0x663, pf=0x1, revision=0x1
[   87.356384] microcode: CPU197 sig=0x663, pf=0x1, revision=0x1
[   87.411347] microcode: CPU198 sig=0x663, pf=0x1, revision=0x1
[   87.469332] microcode: CPU199 sig=0x663, pf=0x1, revision=0x1
[   87.535402] microcode: CPU200 sig=0x663, pf=0x1, revision=0x1
[   87.591886] microcode: CPU201 sig=0x663, pf=0x1, revision=0x1
[   87.651844] microcode: CPU202 sig=0x663, pf=0x1, revision=0x1
[   87.705049] microcode: CPU203 sig=0x663, pf=0x1, revision=0x1
[   87.755225] microcode: CPU204 sig=0x663, pf=0x1, revision=0x1
[   87.806909] microcode: CPU205 sig=0x663, pf=0x1, revision=0x1
[   87.851849] microcode: CPU206 sig=0x663, pf=0x1, revision=0x1
[   87.906425] microcode: CPU207 sig=0x663, pf=0x1, revision=0x1
[   87.963561] microcode: CPU208 sig=0x663, pf=0x1, revision=0x1
[   88.023435] microcode: CPU209 sig=0x663, pf=0x1, revision=0x1
[   88.076195] microcode: CPU210 sig=0x663, pf=0x1, revision=0x1
[   88.132351] microcode: CPU211 sig=0x663, pf=0x1, revision=0x1
[   88.199434] microcode: CPU212 sig=0x663, pf=0x1, revision=0x1
[   88.259646] microcode: CPU213 sig=0x663, pf=0x1, revision=0x1
[   88.315838] microcode: CPU214 sig=0x663, pf=0x1, revision=0x1
[   88.376516] microcode: CPU215 sig=0x663, pf=0x1, revision=0x1
[   88.435194] microcode: CPU216 sig=0x663, pf=0x1, revision=0x1
[   88.490563] microcode: CPU217 sig=0x663, pf=0x1, revision=0x1
[   88.556175] microcode: CPU218 sig=0x663, pf=0x1, revision=0x1
[   88.639502] microcode: CPU219 sig=0x663, pf=0x1, revision=0x1
[   88.708367] microcode: CPU220 sig=0x663, pf=0x1, revision=0x1
[   88.772543] microcode: CPU221 sig=0x663, pf=0x1, revision=0x1
[   88.826491] microcode: CPU222 sig=0x663, pf=0x1, revision=0x1
[   88.893317] microcode: CPU223 sig=0x663, pf=0x1, revision=0x1
[   88.943402] microcode: CPU224 sig=0x663, pf=0x1, revision=0x1
[   88.995044] microcode: CPU225 sig=0x663, pf=0x1, revision=0x1
[   89.052359] microcode: CPU226 sig=0x663, pf=0x1, revision=0x1
[   89.096898] microcode: CPU227 sig=0x663, pf=0x1, revision=0x1
[   89.155630] microcode: CPU228 sig=0x663, pf=0x1, revision=0x1
[   89.216993] microcode: CPU229 sig=0x663, pf=0x1, revision=0x1
[   89.272879] microcode: CPU230 sig=0x663, pf=0x1, revision=0x1
[   89.336496] microcode: CPU231 sig=0x663, pf=0x1, revision=0x1
[   89.382634] microcode: CPU232 sig=0x663, pf=0x1, revision=0x1
[   89.444643] microcode: CPU233 sig=0x663, pf=0x1, revision=0x1
[   89.493595] microcode: CPU234 sig=0x663, pf=0x1, revision=0x1
[   89.551675] microcode: CPU235 sig=0x663, pf=0x1, revision=0x1
[   89.601084] microcode: CPU236 sig=0x663, pf=0x1, revision=0x1
[   89.665280] microcode: CPU237 sig=0x663, pf=0x1, revision=0x1
[   89.730255] microcode: CPU238 sig=0x663, pf=0x1, revision=0x1
[   89.792239] microcode: CPU239 sig=0x663, pf=0x1, revision=0x1
[   89.837199] microcode: CPU240 sig=0x663, pf=0x1, revision=0x1
[   89.897556] microcode: CPU241 sig=0x663, pf=0x1, revision=0x1
[   89.957171] microcode: CPU242 sig=0x663, pf=0x1, revision=0x1
[   90.009898] microcode: CPU243 sig=0x663, pf=0x1, revision=0x1
[   90.079991] microcode: CPU244 sig=0x663, pf=0x1, revision=0x1
[   90.149251] microcode: CPU245 sig=0x663, pf=0x1, revision=0x1
[   90.243148] microcode: CPU246 sig=0x663, pf=0x1, revision=0x1
[   90.339964] microcode: CPU247 sig=0x663, pf=0x1, revision=0x1
[   90.414193] microcode: CPU248 sig=0x663, pf=0x1, revision=0x1
[   90.484587] microcode: CPU249 sig=0x663, pf=0x1, revision=0x1
[   90.548322] microcode: CPU250 sig=0x663, pf=0x1, revision=0x1
[   90.617055] microcode: CPU251 sig=0x663, pf=0x1, revision=0x1
[   90.682249] microcode: CPU252 sig=0x663, pf=0x1, revision=0x1
[   90.740760] microcode: CPU253 sig=0x663, pf=0x1, revision=0x1
[   90.798844] microcode: CPU254 sig=0x663, pf=0x1, revision=0x1
[   90.858954] microcode: CPU255 sig=0x663, pf=0x1, revision=0x1
[   90.918565] microcode: CPU256 sig=0x663, pf=0x1, revision=0x1
[   90.969679] microcode: CPU257 sig=0x663, pf=0x1, revision=0x1
[   91.032451] microcode: CPU258 sig=0x663, pf=0x1, revision=0x1
[   91.085879] microcode: CPU259 sig=0x663, pf=0x1, revision=0x1
[   91.146314] microcode: CPU260 sig=0x663, pf=0x1, revision=0x1
[   91.210725] microcode: CPU261 sig=0x663, pf=0x1, revision=0x1
[   91.263768] microcode: CPU262 sig=0x663, pf=0x1, revision=0x1
[   91.315947] microcode: CPU263 sig=0x663, pf=0x1, revision=0x1
[   91.370642] microcode: CPU264 sig=0x663, pf=0x1, revision=0x1
[   91.432911] microcode: CPU265 sig=0x663, pf=0x1, revision=0x1
[   91.501165] microcode: CPU266 sig=0x663, pf=0x1, revision=0x1
[   91.568599] microcode: CPU267 sig=0x663, pf=0x1, revision=0x1
[   91.619701] microcode: CPU268 sig=0x663, pf=0x1, revision=0x1
[   91.674083] microcode: CPU269 sig=0x663, pf=0x1, revision=0x1
[   91.736809] microcode: CPU270 sig=0x663, pf=0x1, revision=0x1
[   91.806852] microcode: CPU271 sig=0x663, pf=0x1, revision=0x1
[   91.865631] microcode: CPU272 sig=0x663, pf=0x1, revision=0x1
[   91.913617] microcode: CPU273 sig=0x663, pf=0x1, revision=0x1
[   91.970958] microcode: CPU274 sig=0x663, pf=0x1, revision=0x1
[   91.983448] microcode: CPU275 sig=0x663, pf=0x1, revision=0x1
[   92.043951] microcode: CPU276 sig=0x663, pf=0x1, revision=0x1
[   92.103276] microcode: CPU277 sig=0x663, pf=0x1, revision=0x1
[   92.162946] microcode: CPU278 sig=0x663, pf=0x1, revision=0x1
[   92.223698] microcode: CPU279 sig=0x663, pf=0x1, revision=0x1
[   92.290263] microcode: CPU280 sig=0x663, pf=0x1, revision=0x1
[   92.356835] microcode: CPU281 sig=0x663, pf=0x1, revision=0x1
[   92.419950] microcode: CPU282 sig=0x663, pf=0x1, revision=0x1
[   92.480228] microcode: CPU283 sig=0x663, pf=0x1, revision=0x1
[   92.543301] microcode: CPU284 sig=0x663, pf=0x1, revision=0x1
[   92.602925] microcode: CPU285 sig=0x663, pf=0x1, revision=0x1
[   92.662443] microcode: CPU286 sig=0x663, pf=0x1, revision=0x1
[   92.719531] microcode: CPU287 sig=0x663, pf=0x1, revision=0x1
[   92.756848] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[   92.839864] sha1_ssse3: Neither AVX nor AVX2 nor SSSE3 is available/usable.
[   92.909483] sha256_ssse3: Neither AVX nor SSSE3 is available/usable.
[   92.982602] futex hash table entries: 131072 (order: 11, 8388608 bytes)
[   93.033644] Initialise system trusted keyring
[   93.086994] audit: initializing netlink socket (disabled)
[   93.143372] type=2000 audit(1470634139.572:1): initialized
[   93.208575] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[   93.277718] zpool: loaded
[   93.309783] zbud: loaded
[   93.336530] VFS: Disk quotas dquot_6.5.2
[   93.375592] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[   93.445539] msgmni has been set to 7639
[   93.468776] Key type big_key registered
[   93.579262] hrtimer: interrupt took 5724342 ns
[  102.059494] NET: Registered protocol family 38
[  102.101543] Key type asymmetric registered
[  102.146957] Asymmetric key parser 'x509' registered
[  102.192695] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[  102.294652] io scheduler noop registered
[  102.332373] io scheduler deadline registered (default)
[  102.391796] io scheduler cfq registered
[  102.433651] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[  102.490760] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[  102.553157] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[  102.625522] ACPI: Power Button [PWRF]
[  102.663042] GHES: HEST is not enabled!
[  102.704217] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  102.912439] 00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[  102.974073] Non-volatile memory driver v1.3
[  103.017226] Linux agpgart interface v0.103
[  103.053138] crash memory driver: version 1.1
[  103.095281] rdac: device handler registered
[  103.141166] hp_sw: device handler registered
[  103.190279] emc: device handler registered
[  103.225095] alua: device handler registered
[  103.267787] libphy: Fixed MDIO Bus: probed
[  103.316151] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[  103.376917] ehci-pci: EHCI PCI platform driver
[  103.425343] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[  103.478060] ohci-pci: OHCI PCI platform driver
[  103.504410] uhci_hcd: USB Universal Host Controller Interface driver
[  103.577810] usbcore: registered new interface driver usbserial
[  103.631389] usbcore: registered new interface driver usbserial_generic
[  103.712973] usbserial: USB Serial support registered for generic
[  103.771449] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  103.868964] i8042: No controller found
[  103.898947] mousedev: PS/2 mouse device common for all mice
[  103.961241] rtc_cmos 00:00: RTC can wake from S4
[  104.006728] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 56, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  104.072163] rtc_cmos 00:00: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[  104.088934] do_IRQ: 130.97 No irq handler for vector (irq -1)
[  104.209286] cpuidle: using governor menu
[  104.246225] hidraw: raw HID events driver (C) Jiri Kosina
[  104.294219] usbcore: registered new interface driver usbhid
[  104.352874] usbhid: USB HID core driver
[  104.392890] drop_monitor: Initializing network drop monitor service
[  104.452169] TCP: cubic registered
[  104.484676] Initializing XFRM netlink socket
[  104.534997] NET: Registered protocol family 10
[  104.579028] NET: Registered protocol family 17
[  104.619300] Loading compiled-in X.509 certificates
[  104.668323] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80'
[  104.771579] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8'
[  104.882526] Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: bc73c3cee89e5eae994ae50a0db1f0fee3fc0913'
[  104.994780] registered taskstats version 1
[  105.038834] Key type trusted registered
[  105.079380] Key type encrypted registered
[  105.122580] IMA: No TPM chip found, activating TPM-bypass!
[  105.178748] rtc_cmos 00:00: setting system clock to 2016-08-08 05:29:48 UTC (1470634188)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 52, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  105.280752] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  105.348498] Freeing unused kernel memory: 1620k freed
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  105.422372] do_IRQ: 146.113 No irq handler for vector (irq -1)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  105.491704] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[  105.669191] systemd[1]: Detected virtualization kvm.
[  105.728221] systemd[1]: Detected architecture x86-64.
[  105.789844] systemd[1]: Running in initial RAM disk.
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  105.850327] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  105.926390] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  105.982715] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  106.033171] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  106.094204] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  106.158136] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  106.213759] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  135.927277] do_IRQ: 1 callbacks suppressed
[  136.005169] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.076413] do(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.139359] systemd[1]: Set hostname to <localhost.localdomain>.
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  136.232411] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.300082] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.364044] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.432736] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.513535] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.569948] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.616019] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  136.659969] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.301219] do_IRQ: 1 callbacks suppressed
[  166.(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
337809] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.390615] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.390658] systemd[1]: Reached target Timers.
[  166.390674] systemd[1]: Starting Timers.
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  166.552741] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.621928] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.680283] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.739420] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.795853] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.855281] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.918120] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  166.968719] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.040615] do_IRQ: 1 callbacks suppressed
[  197.077887] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.140339] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.209020] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.269182] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.327099] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.385448] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.448693] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.497526] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.561971] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  197.626089] do_IRQ: 146.113 No irq handler for vector (irq -1)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  197.699944] systemd[1]: Created slice -.slice.
[  197.750430] systemd[1]: Starting -.slice.
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  227.826632] do_IRQ: 3 callbacks suppressed
[  227.865640] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  227.937058] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.004287] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.070512] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.128173] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.185530] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.245444] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.311602] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.375317] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  228.431623] do_IRQ: 146.113 No irq handler for vector (irq -1)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  228.497085] systemd[1]: Listening on Journal Socket.
[  228.548061] systemd[1]: Starting Journal Socket.
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  258.614741] do_IRQ: 3 callbacks suppressed
[  258.661931] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  258.721110] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  258.780386] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  258.848379] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  258.911309] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  258.978430] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  259.033263] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  259.099651] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  259.170301] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  259.235126] do_IRQ: 146.113 No irq handler for vector (irq -1)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  259.310248] systemd[1]: Listening on udev Kernel Socket.
[  259.366543] systemd[1]: Starting udev Kernel Socket.
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  289.445714] do_IRQ: 3 callbacks suppressed
[  289.483728] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.537079] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.586525] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.635023] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.693274] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.739295] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.777556] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.824244] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.868530] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  289.923403] do_IRQ: 146.113 No irq handler for vector (irq -1)
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  289.978993] systemd[1]: Created slice System Slice.
[  290.013917] systemd[1]: Starting System Slice.
[  290.048030] systemd[1]: Started dracut ask for additional cmdline parameters.
[  290.103246] systemd[1]: Starting Journal Service...
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 1, IRTE: 49, index: 18
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 2, IRTE: 48, index: 2
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 3, IRTE: 51, index: 3
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 4, IRTE: 113, index: 21
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 5, IRTE: 53, index: 5
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 6, IRTE: 54, index: 6
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 7, IRTE: 55, index: 20
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 8, IRTE: 97, index: 17
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 9, IRTE: 81, index: 16
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 10, IRTE: 58, index: 10
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 11, IRTE: 59, index: 11
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 12, IRTE: 60, index: 19
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 13, IRTE: 61, index: 13
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 14, IRTE: 62, index: 14
(vtd)vtd_interrupt_remap_msi: IOAPIC vector inconsistent: entry: 15, IRTE: 63, index: 15
[  298.278487] do_IRQ: 1 callbacks suppressed
[  298.313202] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.368378] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.421871] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.475943] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.526054] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.571280] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.624504] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.672566] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.723956] do_IRQ: 146.113 No irq handler for vector (irq -1)
[  298.790131] do_IRQ: 146.113 No irq handler for vector (irq -1)

[-- Attachment #3: windows_1vcpu.log --]
[-- Type: text/plain, Size: 10275 bytes --]

***************CMD***************
qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm -M kernel-irqchip=split -serial stdio -bios /home/gao/seabios/seabios/out/bios.bin -smp cpus=1 -hda win8.1.qcow2 -device intel-iommu,intremap=on -machine q35 2>>./log/08.08.2016+13:53:22.log >&2
***************CMD***************
(vtd)vtd_register_types: 
char device redirected to /dev/pts/3 (label compat_monitor0)
(vtd)vtd_realize: 
(vtd)vtd_reset: 
Changing serial settings was 0/0 now 3/0
SeaBIOS (version ?-20160808_104017-g.c)
BUILD: gcc: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4) binutils: version 2.23.52.0.1-55.el7 20130226
No Xen hypervisor found.
vendor 8086 device 29c0
Running on QEMU (q35)
Running on KVM
RamSize: 0x80000000 [cmos]
Relocating init from 0x000da4e0 to 0x7ffac9d0 (size 79264)
Found QEMU fw_cfg
QEMU fw_cfg DMA interface supported
RamBlock: addr 0x0000000000000000 len 0x0000000080000000 [e820]
RamBlock: addr 0x0000000100000000 len 0x0000000080000000 [e820]
Moving pm_base to 0x600
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 6 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c09f
PCI: 32: 00000000c0000000 - 00000000fec00000
PCI: map device bdf=00:02.0  bar 1, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:1f.3  bar 4, addr 0000c040, size 00000040 [io]
PCI: map device bdf=00:1f.2  bar 4, addr 0000c080, size 00000020 [io]
PCI: map device bdf=00:02.0  bar 6, addr feb80000, size 00040000 [mem]
PCI: map device bdf=00:02.0  bar 0, addr febc0000, size 00020000 [mem]
PCI: map device bdf=00:01.0  bar 6, addr febe0000, size 00010000 [mem]
PCI: map device bdf=00:01.0  bar 2, addr febf0000, size 00001000 [mem]
PCI: map device bdf=00:1f.2  bar 5, addr febf1000, size 00001000 [mem]
PCI: map device bdf=00:01.0  bar 0, addr fd000000, size 01000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:29c0
PCI: init bdf=00:01.0 id=1234:1111
PCI: init bdf=00:02.0 id=8086:100e
PCI: init bdf=00:1f.0 id=8086:2918
Q35 LPC init: elcr=00 0c
Using pmtimer, ioport 0x608
PCI: init bdf=00:1f.2 id=8086:2922
PCI: init bdf=00:1f.3 id=8086:2930
PCI: Using 00:01.0 for primary VGA
Found 1 cpu(s) max supported 1 cpu(s)
Copying PIR from 0x7ffbfca8 to 0x000f6b80
Copying MPTABLE from 0x00006df4/7ffa38f0 to 0x000f6aa0
Copying SMBIOS entry point from 0x00006df4 to 0x000f68e0
Scan for VGA option rom
Running option rom at c000:0003
pmm call arg1=0
Turning on vga text mode console
SeaBIOS (version ?-20160808_104017-g.c)
AHCI controller at 00:1f.2, iobase 0xfebf1000, irq 10
Searching bootorder for: /pci@i0cf8/*@1f,2/drive@0/disk@0
AHCI/0: Set transfer mode to UDMA-5
AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk (40960 MiBytes)"
Searching bootorder for: /pci@i0cf8/*@1f,2/drive@2/disk@0
AHCI/2: registering: "DVD/CD [AHCI/2: QEMU DVD-ROM ATAPI-4 DVD/CD]"
PS2 keyboard initialized
Found 1 lpt ports
Found 1 serial ports
All threads complete.
Scan for option roms
Running option rom at c980:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /pci@i0cf8/*@2
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f6840: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=83886080
Running option rom at ca80:0003
Space available for UMB: cd000-ec000, f6400-f67c0
Returned 126976 bytes of ZoneHigh
e820 map has 10 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 000000007ffdf000 = 1 RAM
  4: 000000007ffdf000 - 0000000080000000 = 2 RESERVED
  5: 00000000b0000000 - 00000000c0000000 = 2 RESERVED
  6: 00000000fed1c000 - 00000000fed20000 = 2 RESERVED
  7: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
  8: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
  9: 0000000100000000 - 0000000180000000 = 1 RAM
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00
(vtd)vtd_mem_read: addr 0x10 size 4 val 0xf00f1a
(vtd)vtd_mem_read: addr 0x10 size 4 val 0xf00f1a
(vtd)vtd_mem_read: addr 0x38 size 4 val 0x80000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_read: addr 0x34 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x34 size 4 val 0x0
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_read: addr 0xb8 size 8 val 0x190807
(vtd)vtd_mem_read: addr 0xb8 size 4 val 0x190807
(vtd)vtd_mem_read: addr 0xb8 size 8 val 0x190807
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x0
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x1000000
(vtd)vtd_handle_gcmd_write: value 0x1000000 status 0x0
(vtd)vtd_handle_gcmd_sirtp: set Interrupt Remap Table Pointer
(vtd)vtd_interrupt_remap_table_setup: int remap table addr 0x190000 size 256
(vtd)vtd_mem_read: addr 0x1f size 4 val 0x1
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_write: DMAR_GCMD_REG write addr 0x18, size 4, val 0x5000000
(vtd)vtd_handle_gcmd_write: value 0x5000000 status 0x1000000
(vtd)vtd_handle_gcmd_qie: error: can't enable Queued Invalidation: tail 2
(vtd)vtd_handle_gcmd_sirtp: set Interrupt Remap Table Pointer
(vtd)vtd_interrupt_remap_table_setup: int remap table addr 0x190000 size 256
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000
(vtd)vtd_mem_read: addr 0x1c size 4 val 0x1000000

[-- Attachment #4: Screenshot from 2016-08-08 13-35-46.png --]
[-- Type: image/png, Size: 16982 bytes --]

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
> HI, everyone.
> 
> We have done some tests after merging this patch set into the lastest qemu
> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> some problems we have met.
> 
> 1. We can't boot up a 288 vcpus linux guest with CLI:
> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> The problem exists, even after we only assign 32 vcpus to the linux guest.
> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> The output of qemu and kernel is in attachments. Do you have any idea
> about the problem and how to solve it?

IIUC, we need to wait for Radim's QEMU patches to finally enable 288
vcpus?

Btw, could you please try adding this to the QEMU cmdline when testing
with 32 vcpus:

  -global ioapic.version=0x20

I see that you were running RHEL 7.2 guest with a default e1000. In
that case, we may need to boost ioapic version to 0x20.

Thanks,

-- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Mon, 8 Aug 2016 15:41:23 +0800
Chao Gao <chao.gao@intel.com> wrote:

> HI, everyone.
> 
> We have done some tests after merging this patch set into the lastest qemu
> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> some problems we have met.
> 
> 1. We can't boot up a 288 vcpus linux guest with CLI:
> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> The problem exists, even after we only assign 32 vcpus to the linux guest.
> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> The output of qemu and kernel is in attachments. Do you have any idea
> about the problem and how to solve it?
I don't think we ever looked at "kernel-irqchip=split" only in kernel variant's
been targeted so far.
Radim probably knows better whether it should work or not.

Have you tried with smaller amount of CPUs but with APIC IDs above 254,
like in test below?

[...]

> >Tested with following CLI:
> > QEMU -M q35 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
> >      -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0       \
> >      -bios x2apic_bios.bin

Re: [Qemu-devel] [PATCH for-2.8 16/18] pc: add 'etc/boot-cpus' fw_cfg file for machine with more than 255 CPUs

[Igor Mammedov]

On Sun, 7 Aug 2016 09:02:33 +0300
"Michael S. Tsirkin" <mst@redhat.com> wrote:

> On Fri, Aug 05, 2016 at 11:46:15AM +0200, Igor Mammedov wrote:
> > Currently firmware uses 1 byte at 0x5F offset in RTC CMOS
> > to get number of CPUs present at boot. However 1 byte is
> > not enough to handle more than 255 CPUs.  So add a new
> > fw_cfg file that would allow QEMU to tell it.
> > For compat reasons add file only for machine types that
> > support more than 255 CPUs.
> > 
> > Signed-off-by: Igor Mammedov <imammedo@redhat.com>  
> 
> Do we really need this? Why differentiate between boot
> cpus and others? How about reusing the hotplug
> register, such that it looks just as if all CPUs
> were just added by hotplug?
We could hijack cpu hotplug registers or even better extend it
with new command, like GET_PRESENT_CPUS_COUNT

but then we would need to tell firmware where register
is located which would lead to just adding other fw_cfg
file with register address.
So it's not better (maybe worse) than just adding
etc/boot-cpus fw_cfg file.


> 
> > ---
> >  hw/i386/pc.c | 12 +++++++++++-
> >  1 file changed, 11 insertions(+), 1 deletion(-)
> > 
> > diff --git a/hw/i386/pc.c b/hw/i386/pc.c
> > index c2cd5bd..2b5581a 100644
> > --- a/hw/i386/pc.c
> > +++ b/hw/i386/pc.c
> > @@ -1235,9 +1235,11 @@ void pc_machine_done(Notifier *notifier, void *data)
> >      PCMachineState *pcms = container_of(notifier,
> >                                          PCMachineState, machine_done);
> >      PCIBus *bus = pcms->bus;
> > +    static uint16_t boot_cpus;
> >  
> >      /* set the number of CPUs */
> > -    rtc_set_memory(pcms->rtc, 0x5f, pc_present_cpus_count(pcms) - 1);
> > +    boot_cpus = pc_present_cpus_count(pcms);
> > +    rtc_set_memory(pcms->rtc, 0x5f, boot_cpus - 1);
> >  
> >      if (bus) {
> >          int extra_hosts = 0;
> > @@ -1258,8 +1260,16 @@ void pc_machine_done(Notifier *notifier, void *data)
> >  
> >      acpi_setup();
> >      if (pcms->fw_cfg) {
> > +        MachineClass *mc = MACHINE_GET_CLASS(pcms);
> > +
> >          pc_build_smbios(pcms->fw_cfg);
> >          pc_build_feature_control_file(pcms);
> > +
> > +        if (mc->max_cpus > 255) {
> > +            boot_cpus = cpu_to_le16(boot_cpus);
> > +            fw_cfg_add_file(pcms->fw_cfg, "etc/boot-cpus", &boot_cpus,
> > +                            sizeof(boot_cpus));
> > +        }
> >      }
> >  }
> >  
> > -- 
> > 2.7.4  

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Chao Gao]

[-- Attachment #1: Type: text/plain, Size: 3715 bytes --]

On Mon, Aug 08, 2016 at 11:18:20AM +0200, Igor Mammedov wrote:
>On Mon, 8 Aug 2016 15:41:23 +0800
>Chao Gao <chao.gao@intel.com> wrote:
>
>> HI, everyone.
>> 
>> We have done some tests after merging this patch set into the lastest qemu
>> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
>> some problems we have met.
>> 
>> 1. We can't boot up a 288 vcpus linux guest with CLI:
>> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
>> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
>> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
>> The problem exists, even after we only assign 32 vcpus to the linux guest.
>> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
>> The output of qemu and kernel is in attachments. Do you have any idea
>> about the problem and how to solve it?
>I don't think we ever looked at "kernel-irqchip=split" only in kernel variant's
>been targeted so far.
>Radim probably knows better whether it should work or not.
>
>Have you tried with smaller amount of CPUs but with APIC IDs above 254,
>like in test below?
>
>[...]
>
>> >Tested with following CLI:
>> > QEMU -M q35 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
>> >      -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0       \
>> >      -bios x2apic_bios.bin

I test with CLI:
qemu-system-x86_64 -M q35 \
-enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
-device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 \
-bios bios.bin -hda vdisk.img -serial stdio -m 4096 2>>qemu_and_guest.log >&2

But, I think there should have a cpu with initial apicid >255 
in /proc/cpuinfo. The log(in attachments) shows that the guest kernel 
treats the other cpu as a bad one. What do you think cause the problem?

# cat /proc/interrupts
localhost login:             CPU0       
   0:        125   IO-APIC-edge      timer
   1:        117   IO-APIC-edge      i8042
   4:        382   IO-APIC-edge      serial
   7:          0   IO-APIC-edge      parport0
   8:          1   IO-APIC-edge      rtc0
   9:          0   IO-APIC-fasteoi   acpi
  12:       1661   IO-APIC-edge      i8042
  16:          0   IO-APIC-fasteoi   i801_smbus
  22:         27   IO-APIC-fasteoi   enp0s2
  24:       7310   PCI-MSI-edge      0000:00:1f.2
 NMI:          0   Non-maskable interrupts
 LOC:       6401   Local timer interrupts
 SPU:          0   Spurious interrupts
 PMI:          0   Performance monitoring interrupts
 IWI:       3870   IRQ work interrupts
 RTR:          0   APIC ICR read retries
 RES:          0   Rescheduling interrupts
 CAL:          0   Function call interrupts
 TLB:          0   TLB shootdowns
 TRM:          0   Thermal event interrupts
 THR:          0   Threshold APIC interrupts
 MCE:          0   Machine check exceptions
 MCP:          1   Machine check polls
 ERR:          0
 MIS:          0

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 6
model name      : QEMU Virtual CPU version 2.5+
stepping        : 3
microcode       : 0x1
cpu MHz         : 3591.682
cache size      : 4096 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm rep_good nopl xtopology pni cx16 x2apic hypervisor lahf_lm
bogomips        : 7183.36
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

[-- Attachment #2: qemu_and_guest.log --]
[-- Type: text/plain, Size: 52290 bytes --]

Warning: Number of hotpluggable cpus requested (288) exceeds the recommended cpus supported by KVM (240)
Changing serial settings was 0/0 now 3/0
SeaBIOS (version ?-20160808_104017-g.c)
BUILD: gcc: (GCC) 4.8.5 20150623 (Red Hat 4.8.5-4) binutils: version 2.23.52.0.1-55.el7 20130226
No Xen hypervisor found.
vendor 8086 device 29c0
Running on QEMU (q35)
Running on KVM
RamSize: 0x80000000 [cmos]
Relocating init from 0x000da4e0 to 0x7ffac9d0 (size 79264)
Found QEMU fw_cfg
QEMU fw_cfg DMA interface supported
RamBlock: addr 0x0000000000000000 len 0x0000000080000000 [e820]
RamBlock: addr 0x0000000100000000 len 0x0000000080000000 [e820]
Moving pm_base to 0x600
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 6 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: IO: c000 - c09f
PCI: 32: 00000000c0000000 - 00000000fec00000
PCI: map device bdf=00:02.0  bar 1, addr 0000c000, size 00000040 [io]
PCI: map device bdf=00:1f.3  bar 4, addr 0000c040, size 00000040 [io]
PCI: map device bdf=00:1f.2  bar 4, addr 0000c080, size 00000020 [io]
PCI: map device bdf=00:02.0  bar 6, addr feb80000, size 00040000 [mem]
PCI: map device bdf=00:02.0  bar 0, addr febc0000, size 00020000 [mem]
PCI: map device bdf=00:01.0  bar 6, addr febe0000, size 00010000 [mem]
PCI: map device bdf=00:01.0  bar 2, addr febf0000, size 00001000 [mem]
PCI: map device bdf=00:1f.2  bar 5, addr febf1000, size 00001000 [mem]
PCI: map device bdf=00:01.0  bar 0, addr fd000000, size 01000000 [prefmem]
PCI: init bdf=00:00.0 id=8086:29c0
PCI: init bdf=00:01.0 id=1234:1111
PCI: init bdf=00:02.0 id=8086:100e
PCI: init bdf=00:1f.0 id=8086:2918
Q35 LPC init: elcr=00 0c
Using pmtimer, ioport 0x608
PCI: init bdf=00:1f.2 id=8086:2922
PCI: init bdf=00:1f.3 id=8086:2930
PCI: Using 00:01.0 for primary VGA
handle_smp: apic_id=286
Found 2 cpu(s) max supported 288 cpu(s)
Copying SMBIOS entry point from 0x00006df4 to 0x000f6a40
Scan for VGA option rom
Running option rom at c000:0003
pmm call arg1=0
Turning on vga text mode console
SeaBIOS (version ?-20160808_104017-g.c)
AHCI controller at 00:1f.2, iobase 0xfebf1000, irq 10
Searching bootorder for: /pci@i0cf8/*@1f,2/drive@0/disk@0
AHCI/0: Set transfer mode to UDMA-5
AHCI/0: registering: "AHCI/0: QEMU HARDDISK ATA-7 Hard-Disk (10240 MiBytes)"
Searching bootorder for: /pci@i0cf8/*@1f,2/drive@2/disk@0
AHCI/2: registering: "DVD/CD [AHCI/2: QEMU DVD-ROM ATAPI-4 DVD/CD]"
PS2 keyboard initialized
Found 1 lpt ports
Found 1 serial ports
All threads complete.
Scan for option roms
Running option rom at c980:0003
pmm call arg1=1
pmm call arg1=0
pmm call arg1=1
pmm call arg1=0
Searching bootorder for: /pci@i0cf8/*@2
Searching bootorder for: /rom@genroms/kvmvapic.bin
Searching bootorder for: HALT
drive 0x000f69a0: PCHS=16383/16/63 translation=lba LCHS=1024/255/63 s=20971520
Running option rom at ca80:0003
Space available for UMB: cd000-ec000, f6400-f6920
Returned 126976 bytes of ZoneHigh
e820 map has 10 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 000000007ffdf000 = 1 RAM
  4: 000000007ffdf000 - 0000000080000000 = 2 RESERVED
  5: 00000000b0000000 - 00000000c0000000 = 2 RESERVED
  6: 00000000fed1c000 - 00000000fed20000 = 2 RESERVED
  7: 00000000feffc000 - 00000000ff000000 = 2 RESERVED
  8: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
  9: 0000000100000000 - 0000000180000000 = 1 RAM
enter handle_19:
  NULL
Booting from Hard Disk...
Booting from 0000:7c00
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.10.0-327.el7.x86_64 (mockbuild@x86-034.build.eng.bos.redhat.com) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Oct 29 17:29:29 EDT 2015
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb console=ttyS0,115200 nosoftlockup LANG=en_US.UTF-8
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x000000000009fc00-0x000000000009ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000000f0000-0x00000000000fffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x000000007ffdefff] usable
[    0.000000] BIOS-e820: [mem 0x000000007ffdf000-0x000000007fffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000b0000000-0x00000000bfffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fed1c000-0x00000000fed1ffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000feffc000-0x00000000feffffff] reserved
[    0.000000] BIOS-e820: [mem 0x00000000fffc0000-0x00000000ffffffff] reserved
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000017fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.8 present.
[    0.000000] Hypervisor detected: KVM
[    0.000000] AGP: No AGP bridge found
[    0.000000] e820: last_pfn = 0x180000 max_arch_pfn = 0x400000000
[    0.000000] PAT not supported by CPU.
[    0.000000] x2apic enabled by BIOS, switching to x2apic ops
[    0.000000] e820: last_pfn = 0x7ffdf max_arch_pfn = 0x400000000
[    0.000000] init_memory_mapping: [mem 0x00000000-0x000fffff]
[    0.000000] init_memory_mapping: [mem 0x17fe00000-0x17fffffff]
[    0.000000] init_memory_mapping: [mem 0x17c000000-0x17fdfffff]
[    0.000000] init_memory_mapping: [mem 0x100000000-0x17bffffff]
[    0.000000] init_memory_mapping: [mem 0x00100000-0x7ffdefff]
[    0.000000] RAMDISK: [mem 0x35a4c000-0x36d1dfff]
[    0.000000] ACPI: RSDP 00000000000f6a10 00014 (v00 BOCHS )
[    0.000000] ACPI: RSDT 000000007ffe921a 00034 (v01 BOCHS  BXPCRSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACP 000000007ffe86b4 00074 (v01 BOCHS  BXPCFACP 00000001 BXPC 00000001)
[    0.000000] ACPI: DSDT 000000007ffe0040 08674 (v01 BOCHS  BXPCDSDT 00000001 BXPC 00000001)
[    0.000000] ACPI: FACS 000000007ffe0000 00040
[    0.000000] ACPI: APIC 000000007ffe8728 00A7E (v01 BOCHS  BXPCAPIC 00000001 BXPC 00000001)
[    0.000000] ACPI: HPET 000000007ffe91a6 00038 (v01 BOCHS  BXPCHPET 00000001 BXPC 00000001)
[    0.000000] ACPI: MCFG 000000007ffe91de 0003C (v01 BOCHS  BXPCMCFG 00000001 BXPC 00000001)
[    0.000000] Setting APIC routing to cluster x2apic.
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at [mem 0x0000000000000000-0x000000017fffffff]
[    0.000000] Initmem setup node 0 [mem 0x00000000-0x17fffffff]
[    0.000000]   NODE_DATA [mem 0x17ffd7000-0x17fffdfff]
[    0.000000] Reserving 161MB of memory at 688MB for crashkernel (System RAM: 4095MB)
[    0.000000] kvm-clock: Using msrs 4b564d01 and 4b564d00
[    0.000000] kvm-clock: cpu 0, msr 1:7ff87001, primary cpu clock
[    0.000000] kvm-clock: using sched offset of 2301107794 cycles
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
[    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
[    0.000000]   Normal   [mem 0x100000000-0x17fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x00001000-0x0009efff]
[    0.000000]   node   0: [mem 0x00100000-0x7ffdefff]
[    0.000000]   node   0: [mem 0x100000000-0x17fffffff]
[    0.000000] ACPI: PM-Timer IO Port: 0x608
[    0.000000] ACPI: X2APIC (apic_id[0xff] uid[0xff] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x100] uid[0x100] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x101] uid[0x101] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x102] uid[0x102] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x103] uid[0x103] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x104] uid[0x104] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x105] uid[0x105] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x106] uid[0x106] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x107] uid[0x107] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x108] uid[0x108] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x109] uid[0x109] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10a] uid[0x10a] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10b] uid[0x10b] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10c] uid[0x10c] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10d] uid[0x10d] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10e] uid[0x10e] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x10f] uid[0x10f] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x110] uid[0x110] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x111] uid[0x111] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x112] uid[0x112] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x113] uid[0x113] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x114] uid[0x114] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x115] uid[0x115] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x116] uid[0x116] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x117] uid[0x117] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x118] uid[0x118] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x119] uid[0x119] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11a] uid[0x11a] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11b] uid[0x11b] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11c] uid[0x11c] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11d] uid[0x11d] disabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11e] uid[0x11e] enabled)
[    0.000000] ACPI: X2APIC (apic_id[0x11f] uid[0x11f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x10] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x11] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x12] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x13] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x14] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x15] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x16] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x17] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x18] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x19] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x1a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x1d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x1e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x1f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x20] lapic_id[0x20] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x21] lapic_id[0x21] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x22] lapic_id[0x22] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x23] lapic_id[0x23] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x24] lapic_id[0x24] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x25] lapic_id[0x25] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x26] lapic_id[0x26] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x27] lapic_id[0x27] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x28] lapic_id[0x28] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x29] lapic_id[0x29] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2a] lapic_id[0x2a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2b] lapic_id[0x2b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2c] lapic_id[0x2c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2d] lapic_id[0x2d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2e] lapic_id[0x2e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x2f] lapic_id[0x2f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x30] lapic_id[0x30] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x31] lapic_id[0x31] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x32] lapic_id[0x32] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x33] lapic_id[0x33] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x34] lapic_id[0x34] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x35] lapic_id[0x35] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x36] lapic_id[0x36] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x37] lapic_id[0x37] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x38] lapic_id[0x38] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x39] lapic_id[0x39] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3a] lapic_id[0x3a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3b] lapic_id[0x3b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3c] lapic_id[0x3c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3d] lapic_id[0x3d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3e] lapic_id[0x3e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x3f] lapic_id[0x3f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x40] lapic_id[0x40] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x41] lapic_id[0x41] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x42] lapic_id[0x42] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x43] lapic_id[0x43] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x44] lapic_id[0x44] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x45] lapic_id[0x45] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x46] lapic_id[0x46] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x47] lapic_id[0x47] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x48] lapic_id[0x48] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x49] lapic_id[0x49] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4a] lapic_id[0x4a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4b] lapic_id[0x4b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4c] lapic_id[0x4c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4d] lapic_id[0x4d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4e] lapic_id[0x4e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x4f] lapic_id[0x4f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x50] lapic_id[0x50] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x51] lapic_id[0x51] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x52] lapic_id[0x52] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x53] lapic_id[0x53] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x54] lapic_id[0x54] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x55] lapic_id[0x55] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x56] lapic_id[0x56] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x57] lapic_id[0x57] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x58] lapic_id[0x58] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x59] lapic_id[0x59] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5a] lapic_id[0x5a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5b] lapic_id[0x5b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5c] lapic_id[0x5c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5d] lapic_id[0x5d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5e] lapic_id[0x5e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x5f] lapic_id[0x5f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x60] lapic_id[0x60] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x61] lapic_id[0x61] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x62] lapic_id[0x62] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x63] lapic_id[0x63] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x64] lapic_id[0x64] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x65] lapic_id[0x65] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x66] lapic_id[0x66] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x67] lapic_id[0x67] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x68] lapic_id[0x68] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x69] lapic_id[0x69] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6a] lapic_id[0x6a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6b] lapic_id[0x6b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6c] lapic_id[0x6c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6d] lapic_id[0x6d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6e] lapic_id[0x6e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x6f] lapic_id[0x6f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x70] lapic_id[0x70] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x71] lapic_id[0x71] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x72] lapic_id[0x72] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x73] lapic_id[0x73] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x74] lapic_id[0x74] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x75] lapic_id[0x75] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x76] lapic_id[0x76] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x77] lapic_id[0x77] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x78] lapic_id[0x78] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x79] lapic_id[0x79] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7a] lapic_id[0x7a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7b] lapic_id[0x7b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7c] lapic_id[0x7c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7d] lapic_id[0x7d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7e] lapic_id[0x7e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x7f] lapic_id[0x7f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x80] lapic_id[0x80] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x81] lapic_id[0x81] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x82] lapic_id[0x82] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x83] lapic_id[0x83] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x84] lapic_id[0x84] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x85] lapic_id[0x85] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x86] lapic_id[0x86] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x87] lapic_id[0x87] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x88] lapic_id[0x88] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x89] lapic_id[0x89] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8a] lapic_id[0x8a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8b] lapic_id[0x8b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8c] lapic_id[0x8c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8d] lapic_id[0x8d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8e] lapic_id[0x8e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x8f] lapic_id[0x8f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x90] lapic_id[0x90] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x91] lapic_id[0x91] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x92] lapic_id[0x92] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x93] lapic_id[0x93] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x94] lapic_id[0x94] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x95] lapic_id[0x95] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x96] lapic_id[0x96] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x97] lapic_id[0x97] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x98] lapic_id[0x98] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x99] lapic_id[0x99] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9a] lapic_id[0x9a] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9b] lapic_id[0x9b] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9c] lapic_id[0x9c] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9d] lapic_id[0x9d] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9e] lapic_id[0x9e] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x9f] lapic_id[0x9f] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa0] lapic_id[0xa0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa1] lapic_id[0xa1] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa2] lapic_id[0xa2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa3] lapic_id[0xa3] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa4] lapic_id[0xa4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa5] lapic_id[0xa5] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa6] lapic_id[0xa6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa7] lapic_id[0xa7] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa8] lapic_id[0xa8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xa9] lapic_id[0xa9] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xaa] lapic_id[0xaa] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xab] lapic_id[0xab] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xac] lapic_id[0xac] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xad] lapic_id[0xad] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xae] lapic_id[0xae] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xaf] lapic_id[0xaf] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb0] lapic_id[0xb0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb1] lapic_id[0xb1] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb2] lapic_id[0xb2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb3] lapic_id[0xb3] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb4] lapic_id[0xb4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb5] lapic_id[0xb5] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb6] lapic_id[0xb6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb7] lapic_id[0xb7] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb8] lapic_id[0xb8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xb9] lapic_id[0xb9] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xba] lapic_id[0xba] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbb] lapic_id[0xbb] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbc] lapic_id[0xbc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbd] lapic_id[0xbd] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbe] lapic_id[0xbe] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xbf] lapic_id[0xbf] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc0] lapic_id[0xc0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc1] lapic_id[0xc1] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc2] lapic_id[0xc2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc3] lapic_id[0xc3] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc4] lapic_id[0xc4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc5] lapic_id[0xc5] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc6] lapic_id[0xc6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc7] lapic_id[0xc7] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc8] lapic_id[0xc8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xc9] lapic_id[0xc9] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xca] lapic_id[0xca] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcb] lapic_id[0xcb] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcc] lapic_id[0xcc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcd] lapic_id[0xcd] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xce] lapic_id[0xce] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xcf] lapic_id[0xcf] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd0] lapic_id[0xd0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd1] lapic_id[0xd1] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd2] lapic_id[0xd2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd3] lapic_id[0xd3] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd4] lapic_id[0xd4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd5] lapic_id[0xd5] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd6] lapic_id[0xd6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd7] lapic_id[0xd7] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd8] lapic_id[0xd8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xd9] lapic_id[0xd9] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xda] lapic_id[0xda] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdb] lapic_id[0xdb] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdc] lapic_id[0xdc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdd] lapic_id[0xdd] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xde] lapic_id[0xde] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xdf] lapic_id[0xdf] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe0] lapic_id[0xe0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe1] lapic_id[0xe1] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe2] lapic_id[0xe2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe3] lapic_id[0xe3] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe4] lapic_id[0xe4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe5] lapic_id[0xe5] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe6] lapic_id[0xe6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe7] lapic_id[0xe7] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe8] lapic_id[0xe8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xe9] lapic_id[0xe9] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xea] lapic_id[0xea] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xeb] lapic_id[0xeb] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xec] lapic_id[0xec] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xed] lapic_id[0xed] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xee] lapic_id[0xee] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xef] lapic_id[0xef] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf0] lapic_id[0xf0] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf1] lapic_id[0xf1] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf2] lapic_id[0xf2] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf3] lapic_id[0xf3] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf4] lapic_id[0xf4] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf5] lapic_id[0xf5] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf6] lapic_id[0xf6] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf7] lapic_id[0xf7] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf8] lapic_id[0xf8] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xf9] lapic_id[0xf9] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfa] lapic_id[0xfa] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfb] lapic_id[0xfb] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfc] lapic_id[0xfc] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfd] lapic_id[0xfd] disabled)
[    0.000000] ACPI: LAPIC (acpi_id[0xfe] lapic_id[0xfe] disabled)
[    0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] dfl dfl lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 0, version 17, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 5 global_irq 5 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 10 global_irq 10 high level)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 11 global_irq 11 high level)
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] smpboot: Allowing 288 CPUs, 286 hotplug CPUs
[    0.000000] PM: Registered nosave memory: [mem 0x0009f000-0x0009ffff]
[    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000effff]
[    0.000000] PM: Registered nosave memory: [mem 0x000f0000-0x000fffff]
[    0.000000] PM: Registered nosave memory: [mem 0x7ffdf000-0x7fffffff]
[    0.000000] PM: Registered nosave memory: [mem 0x80000000-0xafffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xb0000000-0xbfffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xc0000000-0xfed1bfff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed1c000-0xfed1ffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfed20000-0xfeffbfff]
[    0.000000] PM: Registered nosave memory: [mem 0xfeffc000-0xfeffffff]
[    0.000000] PM: Registered nosave memory: [mem 0xff000000-0xfffbffff]
[    0.000000] PM: Registered nosave memory: [mem 0xfffc0000-0xffffffff]
[    0.000000] e820: [mem 0xc0000000-0xfed1bfff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on KVM
[    0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:288 nr_cpu_ids:288 nr_node_ids:1
[    0.000000] PERCPU: Embedded 31 pages/cpu @ffff880179200000 s87168 r8192 d31616 u131072
[    0.000000] KVM setup async PF for cpu 0
[    0.000000] kvm-stealtime: cpu 0, msr 17920d240
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 1032040
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-3.10.0-327.el7.x86_64 root=/dev/mapper/rhel-root ro crashkernel=auto rd.lvm.lv=rhel/root rd.lvm.lv=rhel/swap rhgb console=ttyS0,115200 nosoftlockup LANG=en_US.UTF-8
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] AGP: Checking aperture...
[    0.000000] AGP: No AGP bridge found
[    0.000000] Memory: 3825936k/6291456k available (6440k kernel code, 2097676k absent, 367844k reserved, 4266k data, 1620k init)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=288, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=288.
[    0.000000] 	Offload RCU callbacks from all CPUs
[    0.000000] 	Offload RCU callbacks from CPUs: 0-287.
[    0.000000] NR_IRQS:327936 nr_irqs:2728 0
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] allocated 16777216 bytes of page_cgroup
[    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
[    0.000000] tsc: Detected 3591.682 MHz processor
[    0.247261] Calibrating delay loop (skipped) preset value.. 7183.36 BogoMIPS (lpj=3591682)
[    0.247826] pid_max: default: 294912 minimum: 2304
[    0.248298] Security Framework initialized
[    0.248831] SELinux:  Initializing.
[    0.249452] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.250618] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.251381] Mount-cache hash table entries: 4096
[    0.252105] Initializing cgroup subsys memory
[    0.252418] Initializing cgroup subsys devices
[    0.252733] Initializing cgroup subsys freezer
[    0.253096] Initializing cgroup subsys net_cls
[    0.253404] Initializing cgroup subsys blkio
[    0.253687] Initializing cgroup subsys perf_event
[    0.254020] Initializing cgroup subsys hugetlb
[    0.254367] CPU: Physical Processor ID: 0
[    0.254637] CPU: Processor Core ID: 0
[    0.254948] mce: CPU supports 10 MCE banks
[    0.255238] Last level iTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.255238] Last level dTLB entries: 4KB 0, 2MB 0, 4MB 0
[    0.255238] tlb_flushall_shift: 6
[    0.256273] Freeing SMP alternatives: 28k freed
[    0.258532] ACPI: Core revision 20130517
[    0.261318] ACPI: All ACPI Tables successfully acquired
[    0.261703] ftrace: allocating 24585 entries in 97 pages
[    0.276802] Disabling x2apic
[    0.277358] Switched APIC routing to physical flat.
[    0.278342] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.278863] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (fam: 06, model: 06, stepping: 03)
[    0.380978] Performance Events: Broken PMU hardware detected, using software events only.
[    0.381677] Failed to access perfctr msr (MSR c2 is 0)
[    0.386724] smpboot: native_cpu_up: bad cpu 1
[    0.387107] Brought up 1 CPUs
[    0.387364] smpboot: Total of 1 processors activated (7183.36 BogoMIPS)
[    0.388652] devtmpfs: initialized
[    0.390080] EVM: security.selinux
[    0.390343] EVM: security.ima
[    0.390567] EVM: security.capability
[    0.391538] atomic64 test passed for x86-64 platform with CX8 and with SSE
[    0.392200] NET: Registered protocol family 16
[    0.392780] ACPI: bus type PCI registered
[    0.393093] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[    0.393702] PCI: MMCONFIG for domain 0000 [bus 00-ff] at [mem 0xb0000000-0xbfffffff] (base 0xb0000000)
[    0.394398] PCI: MMCONFIG at [mem 0xb0000000-0xbfffffff] reserved in E820
[    0.395214] PCI: Using configuration type 1 for base access
[    0.396567] ACPI: Added _OSI(Module Device)
[    0.396893] ACPI: Added _OSI(Processor Device)
[    0.397227] ACPI: Added _OSI(3.0 _SCP Extensions)
[    0.397583] ACPI: Added _OSI(Processor Aggregator Device)
[    0.403276] ACPI: Interpreter enabled
[    0.403571] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20130517/hwxface-571)
[    0.404275] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20130517/hwxface-571)
[    0.405005] ACPI: (supports S0 S3 S4 S5)
[    0.405301] ACPI: Using IOAPIC for interrupt routing
[    0.405688] PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug
[    0.421935] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[    0.422434] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
[    0.423111] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability]
[    0.423785] PCI host bridge to bus 0000:00
[    0.424100] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.424515] pci_bus 0000:00: root bus resource [io  0x0000-0x0cf7]
[    0.424977] pci_bus 0000:00: root bus resource [io  0x0d00-0xffff]
[    0.425440] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff]
[    0.425951] pci_bus 0000:00: root bus resource [mem 0xc0000000-0xfebfffff]
[    0.456874] pci 0000:00:1f.0: quirk: [io  0x0600-0x067f] claimed by ICH6 ACPI/GPIO/TCO
[    0.479750] ACPI: PCI Interrupt Link [LNKA] (IRQs 5 *10 11)
[    0.480288] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[    0.481141] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[    0.481693] ACPI: PCI Interrupt Link [LNKD] (IRQs 5 10 *11)
[    0.482283] ACPI: PCI Interrupt Link [LNKE] (IRQs 5 *10 11)
[    0.482789] ACPI: PCI Interrupt Link [LNKF] (IRQs 5 *10 11)
[    0.483294] ACPI: PCI Interrupt Link [LNKG] (IRQs 5 10 *11)
[    0.483798] ACPI: PCI Interrupt Link [LNKH] (IRQs 5 10 *11)
[    0.484283] ACPI: PCI Interrupt Link [GSIA] (IRQs *16)
[    0.484702] ACPI: PCI Interrupt Link [GSIB] (IRQs *17)
[    0.485119] ACPI: PCI Interrupt Link [GSIC] (IRQs *18)
[    0.485537] ACPI: PCI Interrupt Link [GSID] (IRQs *19)
[    0.485954] ACPI: PCI Interrupt Link [GSIE] (IRQs *20)
[    0.486373] ACPI: PCI Interrupt Link [GSIF] (IRQs *21)
[    0.486785] ACPI: PCI Interrupt Link [GSIG] (IRQs *22)
[    0.487202] ACPI: PCI Interrupt Link [GSIH] (IRQs *23)
[    0.492040] ACPI: Enabled 2 GPEs in block 00 to 3F
[    0.492516] vgaarb: device added: PCI:0000:00:01.0,decodes=io+mem,owns=io+mem,locks=none
[    0.493122] vgaarb: loaded
[    0.493330] vgaarb: bridge control possible 0000:00:01.0
[    0.493773] SCSI subsystem initialized
[    0.494079] ACPI: bus type USB registered
[    0.494396] usbcore: registered new interface driver usbfs
[    0.494807] usbcore: registered new interface driver hub
[    0.495294] usbcore: registered new device driver usb
[    0.495850] PCI: Using ACPI for IRQ routing
[    0.527139] NetLabel: Initializing
[    0.527417] NetLabel:  domain hash size = 128
[    0.527743] NetLabel:  protocols = UNLABELED CIPSOv4
[    0.528126] NetLabel:  unlabeled traffic allowed by default
[    0.528576] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    0.529110] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.529513] hpet0: 3 comparators, 64-bit 100.000000 MHz counter
[    0.533027] Switching to clocksource kvm-clock
[    0.537671] pnp: PnP ACPI init
[    0.537949] ACPI: bus type PNP registered
[    0.539127] pnp: PnP ACPI: found 6 devices
[    0.539453] ACPI: bus type PNP unregistered
[    0.544898] NET: Registered protocol family 2
[    0.545607] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[    0.546206] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[    0.546846] TCP: Hash tables configured (established 32768 bind 32768)
[    0.547342] TCP: reno registered
[    0.547600] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[    0.548054] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[    0.548975] NET: Registered protocol family 1
[    0.549415] Unpacking initramfs...
[    0.725872] Freeing initrd memory: 19272k freed
[    0.727603] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.728123] software IO TLB [mem 0x7bfdf000-0x7ffdf000] (64MB) mapped at [ffff88007bfdf000-ffff88007ffdefff]
[    0.729134] microcode: CPU0 sig=0x663, pf=0x1, revision=0x1
[    0.729594] microcode: Microcode Update Driver: v2.00 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[    0.730282] sha1_ssse3: Neither AVX nor AVX2 nor SSSE3 is available/usable.
[    0.730816] sha256_ssse3: Neither AVX nor SSSE3 is available/usable.
[    0.731812] futex hash table entries: 131072 (order: 11, 8388608 bytes)
[    0.733058] Initialise system trusted keyring
[    0.733509] audit: initializing netlink socket (disabled)
[    0.733920] type=2000 audit(1470710261.428:1): initialized
[    0.747470] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.748504] zpool: loaded
[    0.748761] zbud: loaded
[    0.749137] VFS: Disk quotas dquot_6.5.2
[    0.749520] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.750244] msgmni has been set to 7510
[    0.750633] Key type big_key registered
[    0.751609] NET: Registered protocol family 38
[    0.751961] Key type asymmetric registered
[    0.752268] Asymmetric key parser 'x509' registered
[    0.752681] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 251)
[    0.753306] io scheduler noop registered
[    0.753618] io scheduler deadline registered (default)
[    0.754023] io scheduler cfq registered
[    0.754366] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.754794] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    0.755395] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[    0.755968] ACPI: Power Button [PWRF]
[    0.756332] GHES: HEST is not enabled!
[    0.756687] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.779197] 00:04: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    0.779868] Non-volatile memory driver v1.3
[    0.780188] Linux agpgart interface v0.103
[    0.780562] crash memory driver: version 1.1
[    0.780896] rdac: device handler registered
[    0.781217] hp_sw: device handler registered
[    0.781550] emc: device handler registered
[    0.781871] alua: device handler registered
[    0.782241] libphy: Fixed MDIO Bus: probed
[    0.782585] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.783068] ehci-pci: EHCI PCI platform driver
[    0.783414] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.783869] ohci-pci: OHCI PCI platform driver
[    0.784199] uhci_hcd: USB Universal Host Controller Interface driver
[    0.784717] usbcore: registered new interface driver usbserial
[    0.785149] usbcore: registered new interface driver usbserial_generic
[    0.785641] usbserial: USB Serial support registered for generic
[    0.786094] i8042: PNP: PS/2 Controller [PNP0303:KBD,PNP0f13:MOU] at 0x60,0x64 irq 1,12
[    0.787138] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.787518] serio: i8042 AUX port at 0x60,0x64 irq 12
[    0.787934] mousedev: PS/2 mouse device common for all mice
[    0.788596] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    0.790235] rtc_cmos 00:00: RTC can wake from S4
[    0.790745] rtc_cmos 00:00: rtc core: registered rtc_cmos as rtc0
[    0.791272] rtc_cmos 00:00: alarms up to one day, y3k, 114 bytes nvram, hpet irqs
[    0.791859] cpuidle: using governor menu
[    0.792182] hidraw: raw HID events driver (C) Jiri Kosina
[    0.792641] usbcore: registered new interface driver usbhid
[    0.793057] usbhid: USB HID core driver
[    0.793364] drop_monitor: Initializing network drop monitor service
[    0.793955] TCP: cubic registered
[    0.794208] Initializing XFRM netlink socket
[    0.794615] NET: Registered protocol family 10
[    0.795302] NET: Registered protocol family 17
[    0.795754] Loading compiled-in X.509 certificates
[    0.796416] Loaded X.509 cert 'Red Hat Enterprise Linux Driver Update Program (key 3): bf57f3e87362bc7229d9f465321773dfd1f77a80'
[    0.797560] Loaded X.509 cert 'Red Hat Enterprise Linux kpatch signing key: 4d38fd864ebe18c5f0b72e3852e2014c3a676fc8'
[    0.798638] Loaded X.509 cert 'Red Hat Enterprise Linux kernel signing key: bc73c3cee89e5eae994ae50a0db1f0fee3fc0913'
[    0.799441] registered taskstats version 1
[    0.800779] Key type trusted registered
[    0.802077] Key type encrypted registered
[    0.803267] IMA: No TPM chip found, activating TPM-bypass!
[    0.804214] rtc_cmos 00:00: setting system clock to 2016-08-09 02:37:41 UTC (1470710261)
[    0.805344] Freeing unused kernel memory: 1620k freed
[    0.808963] systemd[1]: systemd 219 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 -SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
[    0.810321] systemd[1]: Detected virtualization kvm.
[    0.810716] systemd[1]: Detected architecture x86-64.
[    0.811117] systemd[1]: Running in initial RAM disk.

Welcome to ^[[0;34mRed Hat Enterprise Linux Server 7.2 (Maipo) dracut-033-359.el7 (Initramfs)^[[0m!

[    0.812309] systemd[1]: Set hostname to <localhost.localdomain>.
[^[[32m  OK  ^[[0m] Reached target Local File Systems.
[    0.829439] systemd[1]: Reached target Local File Systems.
[    0.829857] systemd[1]: Starting Local File Systems.
[^[[32m  OK  ^[[0m] Created slice -.slice.
[    0.830813] systemd[1]: Created slice -.slice.
[    0.831152] systemd[1]: Starting -.slice.
[^[[32m  OK  ^[[0m] Listening on Journal Socket.
[    0.832011] systemd[1]: Listening on Journal Socket.
[    0.832398] systemd[1]: Starting Journal Socket.
[^[[32m  OK  ^[[0m] Listening on udev Kernel Socket.
[    0.833292] systemd[1]: Listening on udev Kernel Socket.
[    0.833716] systemd[1]: Starting udev Kernel Socket.
[^[[32m  OK  ^[[0m] Listening on udev Control Socket.
[    0.834687] systemd[1]: Listening on udev Control Socket.
[    0.835095] systemd[1]: Starting udev Control Socket.
[^[[32m  OK  ^[[0m] Reached target Sockets.
[    0.835991] systemd[1]: Reached target Sockets.
[    0.836382] systemd[1]: Starting Sockets.
[^[[32m  OK  ^[[0m] Reached target Timers.
[    0.837178] systemd[1]: Reached target Timers.
[    0.837540] systemd[1]: Starting Timers.
[^[[32m  OK  ^[[0m] Reached target Swap.
[    0.838303] systemd[1]: Reached target Swap.
[    0.838673] systemd[1]: Starting Swap.
[^[[32m  OK  ^[[0m] Created slice System Slice.
[    0.839558] systemd[1]: Created slice System Slice.
[    0.839930] systemd[1]: Starting System Slice.
[    0.840286] systemd[1]: Started dracut ask for additional cmdline parameters.
[^[[32m  OK  ^[[0m] Reached target Slices.
[    0.841319] systemd[1]: Reached target Slices.
[    0.841680] systemd[1]: Starting Slices.
[    0.842236] systemd[1]: Starting Journal Service...
         Starting Journal Service...
[    0.843232] systemd[1]: Starting Setup Virtual Console...
         Starting Setup Virtual Console...
[    0.844351] systemd[1]: Starting dracut cmdline hook...
         Starting dracut cmdline hook...
[    0.846624] systemd[1]: Starting Create list of required static device nodes for the current kernel...
         Starting Create list of required st... nodes for the current kernel...
[    0.852244] systemd[1]: Started Load Kernel Modules.
[    0.854111] systemd[1]: Starting Apply Kernel Variables...
         Starting Apply Kernel Variables...
[^[[32m  OK  ^[[0m] Started Create list of required sta...ce nodes for the current kernel.
[    0.860299] systemd[1]: Started Create list of required static device nodes for the current kernel.
[^[[32m  OK  ^[[0m] Started Apply Kernel Variables.
[    0.890984] systemd[1]: Started Apply Kernel Variables.
[^[[32m  OK  ^[[0m] Started Journal Service.
[    0.898430] systemd[1]: Started Journal Service.
         Starting Create Static Device Nodes in /dev...
[^[[32m  OK  ^[[0m] Started Create Static Device Nodes in /dev.
[^[[32m  OK  ^[[0m] Started Setup Virtual Console.
[^[[32m  OK  ^[[0m] Started dracut cmdline hook.
         Starting dracut pre-udev hook...
[    0.981454] device-mapper: uevent: version 1.0.3
[    0.981952] device-mapper: ioctl: 4.33.0-ioctl (2015-8-18) initialised: dm-devel@redhat.com
[^[[32m  OK  ^[[0m] Started dracut pre-udev hook.
         Starting udev Kernel Device Manager...
[^[[32m  OK  ^[[0m] Started udev Kernel Device Manager.
         Starting udev Coldplug all Devices...
         Mounting Configuration File System...
[^[[32m  OK  ^[[0m] Mounted Configuration File System.
[^[[32m  OK  ^[[0m] Started udev Coldplug all Devices.
         Starting Show Plymouth Boot Screen...
[^[[32m  OK  ^[[0m] Reached target System Initialization.
         Starting dracut initqueue hook...
[    1.058048] e1000: Intel(R) PRO/1000 Network Driver - version 7.3.21-k8-NAPI
[    1.058811] e1000: Copyright (c) 1999-2006 Intel Corporation.
[    1.070191] smpboot: native_cpu_up: bad cpu 1
[    1.070949] ACPI: PCI Interrupt Link [GSIG] enabled at IRQ 22
[    1.082621] [drm] Initialized drm 1.1.0 20060810
[    1.402485] e1000 0000:00:02.0 eth0: (PCI:33MHz:32-bit) 52:54:00:12:34:56
[    1.403043] e1000 0000:00:02.0 eth0: Intel(R) PRO/1000 Network Connection
[    1.403947] ACPI: PCI Interrupt Link [GSIA] enabled at IRQ 16
[    1.404728] ahci 0000:00:1f.2: AHCI 0001.0000 32 slots 6 ports 1.5 Gbps 0x3f impl SATA mode
[    1.405365] ahci 0000:00:1f.2: flags: ncq only 
[    1.407947] scsi host0: ahci
[    1.409101] scsi host1: ahci
[    1.409488] scsi host2: ahci
[    1.409770] scsi host3: ahci
[    1.410045] scsi host4: ahci
[    1.410318] scsi host5: ahci
[    1.410580] ata1: SATA max UDMA/133 abar m4096@0xfebf1000 port 0xfebf1100 irq 24
[    1.411127] ata2: SATA max UDMA/133 abar m4096@0xfebf1000 port 0xfebf1180 irq 24
[    1.411683] ata3: SATA max UDMA/133 abar m4096@0xfebf1000 port 0xfebf1200 irq 24
[    1.412229] ata4: SATA max UDMA/133 abar m4096@0xfebf1000 port 0xfebf1280 irq 24
[    1.412807] ata5: SATA max UDMA/133 abar m4096@0xfebf1000 port 0xfebf1300 irq 24
[    1.413359] ata6: SATA max UDMA/133 abar m4096@0xfebf1000 port 0xfebf1380 irq 24
[    1.414347] [drm] Found bochs VGA, ID 0xb0c0.
[    1.414689] [drm] Framebuffer size 16384 kB @ 0xfd000000, mmio @ 0xfebf0000.
[    1.415390] [TTM] Zone  kernel: Available graphics memory: 1923428 kiB
[    1.415943] [TTM] Initializing pool allocator
[    1.416272] [TTM] Initializing DMA pool allocator
[    1.417646] fbcon: bochsdrmfb (fb0) is primary device
[    1.425500] Console: switching to colour frame buffer device 128x48
[    1.427521] bochs-drm 0000:00:01.0: fb0: bochsdrmfb frame buffer device
[    1.428007] bochs-drm 0000:00:01.0: registered panic notifier
[    1.429562] [drm] Initialized bochs-drm 1.0.0 20130925 for 0000:00:01.0 on minor 0
[    1.719582] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    1.720210] ata1.00: ATA-7: QEMU HARDDISK, 2.5+, max UDMA/100
[    1.720704] ata1.00: 20971520 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    1.721213] ata1.00: applying bridge limits
[    1.721688] ata2: SATA link down (SStatus 0 SControl 300)
[    1.722161] ata1.00: configured for UDMA/100
[    1.722608] scsi 0:0:0:0: Direct-Access     ATA      QEMU HARDDISK    2.5+ PQ: 0 ANSI: 5
[    1.730445] tsc: Refined TSC clocksource calibration: 3591.683 MHz
[    1.733490] ata3: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
[    1.734044] ata3.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100
[    1.734521] ata3.00: applying bridge limits
[    1.734934] ata5: SATA link down (SStatus 0 SControl 300)
[    1.735425] ata4: SATA link down (SStatus 0 SControl 300)
[    1.735926] ata3.00: configured for UDMA/100
[    1.736413] scsi 2:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM     2.5+ PQ: 0 ANSI: 5
[    1.737236] ata6: SATA link down (SStatus 0 SControl 300)
[    1.745876] sd 0:0:0:0: [sda] 20971520 512-byte logical blocks: (10.7 GB/10.0 GiB)
[    1.746582] sd 0:0:0:0: [sda] Write Protect is off
[    1.746967] sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    1.748040]  sda: sda1 sda2
[    1.748574] sd 0:0:0:0: [sda] Attached SCSI disk
[    1.753553] sr 2:0:0:0: [sr0] scsi3-mmc drive: 4x/4x cd/rw xa/form2 tray
[    1.754099] cdrom: Uniform CD-ROM driver Revision: 3.20
[    1.768035] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input2
[    1.910347] SGI XFS with ACLs, security attributes, no debug enabled
[    1.912783] XFS (dm-0): Mounting V4 Filesystem
[    1.917419] XFS (dm-0): Ending clean mount
[    2.114648] systemd-journald[662]: Received SIGTERM from PID 1 (systemd).
[    2.130766] type=1404 audit(1470710262.826:2): enforcing=1 old_enforcing=0 auid=4294967295 ses=4294967295
[    2.235877] type=1403 audit(1470710262.931:3): policy loaded auid=4294967295 ses=4294967295
[    2.240306] systemd[1]: Successfully loaded SELinux policy in 109.849ms.
[    2.253373] ip_tables: (C) 2000-2006 Netfilter Core Team
[    2.253882] systemd[1]: Inserted module 'ip_tables'
[    2.266651] systemd[1]: Relabelled /dev and /run in 10.495ms.
[    2.498026] systemd-journald[1045]: Received request to flush runtime journal from PID 1
[    3.045477] parport_pc 00:03: reported by Plug and Play ACPI
[    3.046045] parport0: PC-style at 0x378, irq 7 [PCSPP,TRISTATE]
[    3.049900] smpboot: native_cpu_up: bad cpu 1
[    3.061297] lpc_ich 0000:00:1f.0: I/O space for GPIO uninitialized
[    3.077500] i801_smbus 0000:00:1f.3: SMBus using PCI interrupt
[    3.079529] input: PC Speaker as /devices/platform/pcspkr/input/input3
[    3.090031] sd 0:0:0:0: Attached scsi generic sg0 type 0
[    3.090844] sr 2:0:0:0: Attached scsi generic sg1 type 5
[    3.156057] ppdev: user-space parallel port driver
[    3.169195] iTCO_vendor_support: vendor-support=0
[    3.171266] iTCO_wdt: Intel TCO WatchDog Timer Driver v1.11
[    3.171803] iTCO_wdt: Found a ICH9 TCO device (Version=2, TCOBASE=0x0660)
[    3.172990] iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0)
[    3.206608] Adding 1048572k swap on /dev/mapper/rhel-swap.  Priority:-1 extents:1 across:1048572k FS
[    3.254000] XFS (sda1): Mounting V4 Filesystem
[    3.471095] XFS (sda1): Ending clean mount
[    3.604506] type=1305 audit(1470710264.300:4): audit_pid=1132 old=0 auid=4294967295 ses=4294967295 subj=system_u:system_r:auditd_t:s0 res=1
[    3.920859] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[    3.946214] ip6_tables: (C) 2000-2006 Netfilter Core Team
[    4.045905] Ebtables v2.0 registered
[    4.071260] Bridge firewalling registered
[    4.217804] IPv6: ADDRCONF(NETDEV_UP): enp0s2: link is not ready
[    4.220019] e1000: enp0s2 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX

Red Hat Enterprise Linux Server 7.2 (Maipo)
Kernel 3.10.0-327.el7.x86_64 on an x86_64

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Chao Gao]

On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
>On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
>> HI, everyone.
>> 
>> We have done some tests after merging this patch set into the lastest qemu
>> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
>> some problems we have met.
>> 
>> 1. We can't boot up a 288 vcpus linux guest with CLI:
>> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
>> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
>> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
>> The problem exists, even after we only assign 32 vcpus to the linux guest.
>> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
>> The output of qemu and kernel is in attachments. Do you have any idea
>> about the problem and how to solve it?
>
>IIUC, we need to wait for Radim's QEMU patches to finally enable 288
>vcpus?
>
>Btw, could you please try adding this to the QEMU cmdline when testing
>with 32 vcpus:
>
>  -global ioapic.version=0x20
>
>I see that you were running RHEL 7.2 guest with a default e1000. In
>that case, we may need to boost ioapic version to 0x20.

It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
is in kernel, intremap can only handle situations that vcpus number is less than 
physical cpus'. Do you think it's right? 

Thanks,
-Chao

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
> On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
> >On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
> >> HI, everyone.
> >> 
> >> We have done some tests after merging this patch set into the lastest qemu
> >> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> >> some problems we have met.
> >> 
> >> 1. We can't boot up a 288 vcpus linux guest with CLI:
> >> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> >> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> >> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> >> The problem exists, even after we only assign 32 vcpus to the linux guest.
> >> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> >> The output of qemu and kernel is in attachments. Do you have any idea
> >> about the problem and how to solve it?
> >
> >IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> >vcpus?
> >
> >Btw, could you please try adding this to the QEMU cmdline when testing
> >with 32 vcpus:
> >
> >  -global ioapic.version=0x20
> >
> >I see that you were running RHEL 7.2 guest with a default e1000. In
> >that case, we may need to boost ioapic version to 0x20.
> 
> It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
> or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
> is in kernel, intremap can only handle situations that vcpus number is less than 
> physical cpus'. Do you think it's right? 

I don't think so. Vcpu number should not be related to host cpu
numbers.

I think the problem is with x2apic. Currently, x2apic is enabled in
vIOMMU when kernel irqchip is used. This is problematic, since
actually we are throughing away dest_id[31:8] without Radim's patches,
meanwhile I see that by default x2apic is using cluster mode.

In cluster mode, 8 bits will possibly not suffice (I think the reason
why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
we'll have trouble if we have more than one cluster).

To temporarily solve your issue, you should not only need "-global
ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
to you guest kernel boot parameter, to force guest boot with x2apic
physical mode (not cluster mode). Though this can only work for <255
vcpus. IMHO we may still need to wait for Radim's patches to test >255
case.

Thanks.

-- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
> On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
> > On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
> > >On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
> > >> HI, everyone.
> > >> 
> > >> We have done some tests after merging this patch set into the lastest qemu
> > >> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> > >> some problems we have met.
> > >> 
> > >> 1. We can't boot up a 288 vcpus linux guest with CLI:
> > >> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> > >> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> > >> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> > >> The problem exists, even after we only assign 32 vcpus to the linux guest.
> > >> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> > >> The output of qemu and kernel is in attachments. Do you have any idea
> > >> about the problem and how to solve it?
> > >
> > >IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> > >vcpus?
> > >
> > >Btw, could you please try adding this to the QEMU cmdline when testing
> > >with 32 vcpus:
> > >
> > >  -global ioapic.version=0x20
> > >
> > >I see that you were running RHEL 7.2 guest with a default e1000. In
> > >that case, we may need to boost ioapic version to 0x20.
> > 
> > It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
> > or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
> > is in kernel, intremap can only handle situations that vcpus number is less than 
> > physical cpus'. Do you think it's right? 
> 
> I don't think so. Vcpu number should not be related to host cpu
> numbers.
> 
> I think the problem is with x2apic. Currently, x2apic is enabled in
> vIOMMU when kernel irqchip is used. This is problematic, since
> actually we are throughing away dest_id[31:8] without Radim's patches,
> meanwhile I see that by default x2apic is using cluster mode.
> 
> In cluster mode, 8 bits will possibly not suffice (I think the reason
> why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
> we'll have trouble if we have more than one cluster).
> 
> To temporarily solve your issue, you should not only need "-global
> ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
> to you guest kernel boot parameter, to force guest boot with x2apic
> physical mode (not cluster mode). Though this can only work for <255
> vcpus. IMHO we may still need to wait for Radim's patches to test >255
> case.

Not sure whether we should temporarily disable EIM by default for now
(provide an extra flag to optionally enable it)? Since it might break
guests with >17 vcpus.

CC Jan as well.

-- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Jan Kiszka]

[-- Attachment #1: Type: text/plain, Size: 3160 bytes --]

On 2016-08-09 08:24, Peter Xu wrote:
> On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
>> On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
>>> On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
>>>> On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
>>>>> HI, everyone.
>>>>>
>>>>> We have done some tests after merging this patch set into the lastest qemu
>>>>> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
>>>>> some problems we have met.
>>>>>
>>>>> 1. We can't boot up a 288 vcpus linux guest with CLI:
>>>>> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
>>>>> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
>>>>> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
>>>>> The problem exists, even after we only assign 32 vcpus to the linux guest.
>>>>> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
>>>>> The output of qemu and kernel is in attachments. Do you have any idea
>>>>> about the problem and how to solve it?
>>>>
>>>> IIUC, we need to wait for Radim's QEMU patches to finally enable 288
>>>> vcpus?
>>>>
>>>> Btw, could you please try adding this to the QEMU cmdline when testing
>>>> with 32 vcpus:
>>>>
>>>>  -global ioapic.version=0x20
>>>>
>>>> I see that you were running RHEL 7.2 guest with a default e1000. In
>>>> that case, we may need to boost ioapic version to 0x20.
>>>
>>> It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
>>> or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
>>> is in kernel, intremap can only handle situations that vcpus number is less than 
>>> physical cpus'. Do you think it's right? 
>>
>> I don't think so. Vcpu number should not be related to host cpu
>> numbers.
>>
>> I think the problem is with x2apic. Currently, x2apic is enabled in
>> vIOMMU when kernel irqchip is used. This is problematic, since
>> actually we are throughing away dest_id[31:8] without Radim's patches,
>> meanwhile I see that by default x2apic is using cluster mode.
>>
>> In cluster mode, 8 bits will possibly not suffice (I think the reason
>> why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
>> we'll have trouble if we have more than one cluster).
>>
>> To temporarily solve your issue, you should not only need "-global
>> ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
>> to you guest kernel boot parameter, to force guest boot with x2apic
>> physical mode (not cluster mode). Though this can only work for <255
>> vcpus. IMHO we may still need to wait for Radim's patches to test >255
>> case.
> 
> Not sure whether we should temporarily disable EIM by default for now
> (provide an extra flag to optionally enable it)? Since it might break
> guests with >17 vcpus.
> 
> CC Jan as well.

A switch for EIM would be fine for me if it helps.

To my understanding, the issue will be gone with an enhance KVM
interface that we can then also detect via some cap (to flip the default
again)?

Jan


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 181 bytes --]

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Tue, Aug 09, 2016 at 08:33:13AM +0200, Jan Kiszka wrote:
> On 2016-08-09 08:24, Peter Xu wrote:
> > On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
> >> On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
> >>> On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
> >>>> On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
> >>>>> HI, everyone.
> >>>>>
> >>>>> We have done some tests after merging this patch set into the lastest qemu
> >>>>> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> >>>>> some problems we have met.
> >>>>>
> >>>>> 1. We can't boot up a 288 vcpus linux guest with CLI:
> >>>>> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> >>>>> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> >>>>> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> >>>>> The problem exists, even after we only assign 32 vcpus to the linux guest.
> >>>>> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> >>>>> The output of qemu and kernel is in attachments. Do you have any idea
> >>>>> about the problem and how to solve it?
> >>>>
> >>>> IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> >>>> vcpus?
> >>>>
> >>>> Btw, could you please try adding this to the QEMU cmdline when testing
> >>>> with 32 vcpus:
> >>>>
> >>>>  -global ioapic.version=0x20
> >>>>
> >>>> I see that you were running RHEL 7.2 guest with a default e1000. In
> >>>> that case, we may need to boost ioapic version to 0x20.
> >>>
> >>> It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
> >>> or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
> >>> is in kernel, intremap can only handle situations that vcpus number is less than 
> >>> physical cpus'. Do you think it's right? 
> >>
> >> I don't think so. Vcpu number should not be related to host cpu
> >> numbers.
> >>
> >> I think the problem is with x2apic. Currently, x2apic is enabled in
> >> vIOMMU when kernel irqchip is used. This is problematic, since
> >> actually we are throughing away dest_id[31:8] without Radim's patches,
> >> meanwhile I see that by default x2apic is using cluster mode.
> >>
> >> In cluster mode, 8 bits will possibly not suffice (I think the reason
> >> why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
> >> we'll have trouble if we have more than one cluster).
> >>
> >> To temporarily solve your issue, you should not only need "-global
> >> ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
> >> to you guest kernel boot parameter, to force guest boot with x2apic
> >> physical mode (not cluster mode). Though this can only work for <255
> >> vcpus. IMHO we may still need to wait for Radim's patches to test >255
> >> case.
> > 
> > Not sure whether we should temporarily disable EIM by default for now
> > (provide an extra flag to optionally enable it)? Since it might break
> > guests with >17 vcpus.
> > 
> > CC Jan as well.
> 
> A switch for EIM would be fine for me if it helps.
> 
> To my understanding, the issue will be gone with an enhance KVM
> interface that we can then also detect via some cap (to flip the default
> again)?

Would you help explain how to do it?

Btw, if we have that switch, the default can go back to EIM mode along
with Radim's future patches.

Thanks,

-- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Tue, 9 Aug 2016 11:23:58 +0800
Chao Gao <chao.gao@intel.com> wrote:

> On Mon, Aug 08, 2016 at 11:18:20AM +0200, Igor Mammedov wrote:
> >On Mon, 8 Aug 2016 15:41:23 +0800
> >Chao Gao <chao.gao@intel.com> wrote:
> >  
> >> HI, everyone.
> >> 
> >> We have done some tests after merging this patch set into the lastest qemu
> >> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> >> some problems we have met.
> >> 
> >> 1. We can't boot up a 288 vcpus linux guest with CLI:
> >> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> >> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> >> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> >> The problem exists, even after we only assign 32 vcpus to the linux guest.
> >> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> >> The output of qemu and kernel is in attachments. Do you have any idea
> >> about the problem and how to solve it?  
> >I don't think we ever looked at "kernel-irqchip=split" only in kernel variant's
> >been targeted so far.
> >Radim probably knows better whether it should work or not.
> >
> >Have you tried with smaller amount of CPUs but with APIC IDs above 254,
> >like in test below?
> >
> >[...]
> >  
> >> >Tested with following CLI:
> >> > QEMU -M q35 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
> >> >      -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0       \
> >> >      -bios x2apic_bios.bin  
> 
> I test with CLI:
> qemu-system-x86_64 -M q35 \
> -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
> -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 \
> -bios bios.bin -hda vdisk.img -serial stdio -m 4096 2>>qemu_and_guest.log >&2
> 
> But, I think there should have a cpu with initial apicid >255 
> in /proc/cpuinfo. The log(in attachments) shows that the guest kernel 
> treats the other cpu as a bad one. What do you think cause the problem?
Bad cpu happens because of following:
[    0.319911] x2apic: IRQ remapping doesn't support X2APIC mode
[    0.321427] x2apic disabled

if I recall correctly Radim gave me a hack/patch to workaround it,
and that's what I've been testing with.

Anyway from log it looks like CPU with x2apic is there and seabios
initialized it correctly. Missing part is working irq remapping.



> 
> # cat /proc/interrupts
> localhost login:             CPU0       
>    0:        125   IO-APIC-edge      timer
>    1:        117   IO-APIC-edge      i8042
>    4:        382   IO-APIC-edge      serial
>    7:          0   IO-APIC-edge      parport0
>    8:          1   IO-APIC-edge      rtc0
>    9:          0   IO-APIC-fasteoi   acpi
>   12:       1661   IO-APIC-edge      i8042
>   16:          0   IO-APIC-fasteoi   i801_smbus
>   22:         27   IO-APIC-fasteoi   enp0s2
>   24:       7310   PCI-MSI-edge      0000:00:1f.2
>  NMI:          0   Non-maskable interrupts
>  LOC:       6401   Local timer interrupts
>  SPU:          0   Spurious interrupts
>  PMI:          0   Performance monitoring interrupts
>  IWI:       3870   IRQ work interrupts
>  RTR:          0   APIC ICR read retries
>  RES:          0   Rescheduling interrupts
>  CAL:          0   Function call interrupts
>  TLB:          0   TLB shootdowns
>  TRM:          0   Thermal event interrupts
>  THR:          0   Threshold APIC interrupts
>  MCE:          0   Machine check exceptions
>  MCP:          1   Machine check polls
>  ERR:          0
>  MIS:          0
> 
> # cat /proc/cpuinfo
> processor       : 0
> vendor_id       : GenuineIntel
> cpu family      : 6
> model           : 6
> model name      : QEMU Virtual CPU version 2.5+
> stepping        : 3
> microcode       : 0x1
> cpu MHz         : 3591.682
> cache size      : 4096 KB
> physical id     : 0
> siblings        : 1
> core id         : 0
> cpu cores       : 1
> apicid          : 0
> initial apicid  : 0
> fpu             : yes
> fpu_exception   : yes
> cpuid level     : 13
> wp              : yes
> flags           : fpu de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pse36 clflush mmx fxsr sse sse2 ht syscall nx lm rep_good nopl xtopology pni cx16 x2apic hypervisor lahf_lm
> bogomips        : 7183.36
> clflush size    : 64
> cache_alignment : 64
> address sizes   : 40 bits physical, 48 bits virtual
> power management:

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Chao Gao]

On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
>On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
>> On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
>> >On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
>> >> HI, everyone.
>> >> 
>> >> We have done some tests after merging this patch set into the lastest qemu
>> >> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
>> >> some problems we have met.
>> >> 
>> >> 1. We can't boot up a 288 vcpus linux guest with CLI:
>> >> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
>> >> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
>> >> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
>> >> The problem exists, even after we only assign 32 vcpus to the linux guest.
>> >> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
>> >> The output of qemu and kernel is in attachments. Do you have any idea
>> >> about the problem and how to solve it?
>> >
>> >IIUC, we need to wait for Radim's QEMU patches to finally enable 288
>> >vcpus?
>> >
>> >Btw, could you please try adding this to the QEMU cmdline when testing
>> >with 32 vcpus:
>> >
>> >  -global ioapic.version=0x20
>> >
>> >I see that you were running RHEL 7.2 guest with a default e1000. In
>> >that case, we may need to boost ioapic version to 0x20.
>> 
>> It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
>> or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
>> is in kernel, intremap can only handle situations that vcpus number is less than 
>> physical cpus'. Do you think it's right? 
>
>I don't think so. Vcpu number should not be related to host cpu
>numbers.
>
>I think the problem is with x2apic. Currently, x2apic is enabled in
>vIOMMU when kernel irqchip is used. This is problematic, since
>actually we are throughing away dest_id[31:8] without Radim's patches,
>meanwhile I see that by default x2apic is using cluster mode.
>
>In cluster mode, 8 bits will possibly not suffice (I think the reason
>why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
>we'll have trouble if we have more than one cluster).
>
>To temporarily solve your issue, you should not only need "-global
>ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
>to you guest kernel boot parameter, to force guest boot with x2apic
>physical mode (not cluster mode). Though this can only work for <255
>vcpus. IMHO we may still need to wait for Radim's patches to test >255
>case.

ok, after adding "x2apic_phys" to guest kernel boot parameter, I 
boot up a 288(yes, 288) vcpus guest successfully with command
qemu-system-x86_64 -boot c -m 4096 -sdl --enable-kvm \
-M kernel-irqchip=split -bios bios.bin -smp cpus=288 -hda vdisk.img \
-device intel-iommu,intremap=on -machine q35
Also, I can see interrupts on those cpu with inital apicid>255 from /proc/cpuinfo and /proc/interrupts. 

Thanks.
-Chao

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Mon, 8 Aug 2016 16:57:14 +0800
Peter Xu <peterx@redhat.com> wrote:

> On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
> > HI, everyone.
> > 
> > We have done some tests after merging this patch set into the lastest qemu
> > master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> > some problems we have met.
> > 
> > 1. We can't boot up a 288 vcpus linux guest with CLI:
> > qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> > -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> > -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> > The problem exists, even after we only assign 32 vcpus to the linux guest.
> > Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> > The output of qemu and kernel is in attachments. Do you have any idea
> > about the problem and how to solve it?  
> 
> IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> vcpus?
> 
> Btw, could you please try adding this to the QEMU cmdline when testing
> with 32 vcpus:
> 
>   -global ioapic.version=0x20
> 
> I see that you were running RHEL 7.2 guest with a default e1000. In
> that case, we may need to boost ioapic version to 0x20.
> 
> Thanks,
> 
> -- peterx

Peter,

Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:

./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72


[    0.350669] smpboot: Max logical packages: 9
[    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
[    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
[    0.354627] DMAR: Host address width 39
[    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
[    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
[    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
[    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
[    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
[    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
[    0.369373] PGD 0 
[    0.370258] Oops: 0002 [#1] SMP
[    0.371140] Modules linked in:
[    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
[    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
[    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
[    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
[    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
[    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
[    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
[    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
[    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
[    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
[    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
[    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
[    0.392114] Stack:
[    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
[    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
[    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
[    0.399695] Call Trace:
[    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
[    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
[    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
[    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
[    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
[    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
[    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
[    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
[    0.418516]  RSP <ffff880039adbe28>
[    0.419461] CR2: 0000000000000000
[    0.420386] ---[ end trace f68728a0d3053b52 ]---

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Radim Krčmář]

2016-08-09 15:09+0800, Peter Xu:
> On Tue, Aug 09, 2016 at 08:33:13AM +0200, Jan Kiszka wrote:
>> On 2016-08-09 08:24, Peter Xu wrote:
>> > On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
>> >> On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
>> >>> On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
>> >>>> On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
>> >>>>> HI, everyone.
>> >>>>>
>> >>>>> We have done some tests after merging this patch set into the lastest qemu
>> >>>>> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
>> >>>>> some problems we have met.
>> >>>>>
>> >>>>> 1. We can't boot up a 288 vcpus linux guest with CLI:
>> >>>>> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
>> >>>>> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
>> >>>>> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
>> >>>>> The problem exists, even after we only assign 32 vcpus to the linux guest.
>> >>>>> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
>> >>>>> The output of qemu and kernel is in attachments. Do you have any idea
>> >>>>> about the problem and how to solve it?
>> >>>>
>> >>>> IIUC, we need to wait for Radim's QEMU patches to finally enable 288
>> >>>> vcpus?
>> >>>>
>> >>>> Btw, could you please try adding this to the QEMU cmdline when testing
>> >>>> with 32 vcpus:
>> >>>>
>> >>>>  -global ioapic.version=0x20
>> >>>>
>> >>>> I see that you were running RHEL 7.2 guest with a default e1000. In
>> >>>> that case, we may need to boost ioapic version to 0x20.
>> >>>
>> >>> It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
>> >>> or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
>> >>> is in kernel, intremap can only handle situations that vcpus number is less than 
>> >>> physical cpus'. Do you think it's right? 
>> >>
>> >> I don't think so. Vcpu number should not be related to host cpu
>> >> numbers.
>> >>
>> >> I think the problem is with x2apic. Currently, x2apic is enabled in
>> >> vIOMMU when kernel irqchip is used. This is problematic, since
>> >> actually we are throughing away dest_id[31:8] without Radim's patches,
>> >> meanwhile I see that by default x2apic is using cluster mode.
>> >>
>> >> In cluster mode, 8 bits will possibly not suffice (I think the reason
>> >> why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
>> >> we'll have trouble if we have more than one cluster).
>> >>
>> >> To temporarily solve your issue, you should not only need "-global
>> >> ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
>> >> to you guest kernel boot parameter, to force guest boot with x2apic
>> >> physical mode (not cluster mode). Though this can only work for <255
>> >> vcpus. IMHO we may still need to wait for Radim's patches to test >255
>> >> case.
>> > 
>> > Not sure whether we should temporarily disable EIM by default for now
>> > (provide an extra flag to optionally enable it)? Since it might break
>> > guests with >17 vcpus.
>> > 
>> > CC Jan as well.
>> 
>> A switch for EIM would be fine for me if it helps.
>> 
>> To my understanding, the issue will be gone with an enhance KVM
>> interface that we can then also detect via some cap (to flip the default
>> again)?
> 
> Would you help explain how to do it?
> 
> Btw, if we have that switch, the default can go back to EIM mode along
> with Radim's future patches.

I will post patches today as the feature made it upstream.

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Radim Krčmář]

2016-08-09 16:19+0800, Chao Gao:
> On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
>>On Tue, Aug 09, 2016 at 12:33:17PM +0800, Chao Gao wrote:
>>> On Mon, Aug 08, 2016 at 04:57:14PM +0800, Peter Xu wrote:
>>> >On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
>>> >> HI, everyone.
>>> >> 
>>> >> We have done some tests after merging this patch set into the lastest qemu
>>> >> master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
>>> >> some problems we have met.
>>> >> 
>>> >> 1. We can't boot up a 288 vcpus linux guest with CLI:
>>> >> qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
>>> >> -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
>>> >> -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
>>> >> The problem exists, even after we only assign 32 vcpus to the linux guest.
>>> >> Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
>>> >> The output of qemu and kernel is in attachments. Do you have any idea
>>> >> about the problem and how to solve it?
>>> >
>>> >IIUC, we need to wait for Radim's QEMU patches to finally enable 288
>>> >vcpus?
>>> >
>>> >Btw, could you please try adding this to the QEMU cmdline when testing
>>> >with 32 vcpus:
>>> >
>>> >  -global ioapic.version=0x20
>>> >
>>> >I see that you were running RHEL 7.2 guest with a default e1000. In
>>> >that case, we may need to boost ioapic version to 0x20.
>>> 
>>> It doesn't work. My host machine has 16 cpus. When I assign 4 or 8 vcpus to the guest
>>> or 255 vcpus but set "kernel-irqchip=off", the guest work well. Maybe when irqchip
>>> is in kernel, intremap can only handle situations that vcpus number is less than 
>>> physical cpus'. Do you think it's right? 
>>
>>I don't think so. Vcpu number should not be related to host cpu
>>numbers.
>>
>>I think the problem is with x2apic. Currently, x2apic is enabled in
>>vIOMMU when kernel irqchip is used. This is problematic, since
>>actually we are throughing away dest_id[31:8] without Radim's patches,
>>meanwhile I see that by default x2apic is using cluster mode.
>>
>>In cluster mode, 8 bits will possibly not suffice (I think the reason
>>why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
>>we'll have trouble if we have more than one cluster).
>>
>>To temporarily solve your issue, you should not only need "-global
>>ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
>>to you guest kernel boot parameter, to force guest boot with x2apic
>>physical mode (not cluster mode). Though this can only work for <255
>>vcpus. IMHO we may still need to wait for Radim's patches to test >255
>>case.
> 
> ok, after adding "x2apic_phys" to guest kernel boot parameter, I 
> boot up a 288(yes, 288) vcpus guest successfully with command
> qemu-system-x86_64 -boot c -m 4096 -sdl --enable-kvm \
> -M kernel-irqchip=split -bios bios.bin -smp cpus=288 -hda vdisk.img \
> -device intel-iommu,intremap=on -machine q35
> Also, I can see interrupts on those cpu with inital apicid>255 from /proc/cpuinfo and /proc/interrupts. 

Great, thanks for testing!
Only IPIs will be correctly delivered to apic_id > 255 without few more
patches on the QEMU side, though.

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Tue, Aug 09, 2016 at 10:28:41AM +0200, Igor Mammedov wrote:
> On Mon, 8 Aug 2016 16:57:14 +0800
> Peter Xu <peterx@redhat.com> wrote:
> 
> > On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:
> > > HI, everyone.
> > > 
> > > We have done some tests after merging this patch set into the lastest qemu
> > > master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> > > some problems we have met.
> > > 
> > > 1. We can't boot up a 288 vcpus linux guest with CLI:
> > > qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> > > -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> > > -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> > > The problem exists, even after we only assign 32 vcpus to the linux guest.
> > > Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> > > The output of qemu and kernel is in attachments. Do you have any idea
> > > about the problem and how to solve it?  
> > 
> > IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> > vcpus?
> > 
> > Btw, could you please try adding this to the QEMU cmdline when testing
> > with 32 vcpus:
> > 
> >   -global ioapic.version=0x20
> > 
> > I see that you were running RHEL 7.2 guest with a default e1000. In
> > that case, we may need to boost ioapic version to 0x20.
> > 
> > Thanks,
> > 
> > -- peterx
> 
> Peter,
> 
> Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> 
> ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> 
> 
> [    0.350669] smpboot: Max logical packages: 9
> [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> [    0.354627] DMAR: Host address width 39
> [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> [    0.369373] PGD 0 
> [    0.370258] Oops: 0002 [#1] SMP
> [    0.371140] Modules linked in:
> [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> [    0.392114] Stack:
> [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> [    0.399695] Call Trace:
> [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> [    0.418516]  RSP <ffff880039adbe28>
> [    0.419461] CR2: 0000000000000000
> [    0.420386] ---[ end trace f68728a0d3053b52 ]---

Hi, Igor,

Thanks for the feedback!

Not sure whether this patch can fix it (since they looks alike):

  https://lkml.org/lkml/2016/8/4/345

CC Luiz.

Thanks,

-- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Luiz Capitulino]

On Tue, 9 Aug 2016 21:35:04 +0800
Peter Xu <peterx@redhat.com> wrote:

> On Tue, Aug 09, 2016 at 10:28:41AM +0200, Igor Mammedov wrote:
> > On Mon, 8 Aug 2016 16:57:14 +0800
> > Peter Xu <peterx@redhat.com> wrote:
> >   
> > > On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:  
> > > > HI, everyone.
> > > > 
> > > > We have done some tests after merging this patch set into the lastest qemu
> > > > master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> > > > some problems we have met.
> > > > 
> > > > 1. We can't boot up a 288 vcpus linux guest with CLI:
> > > > qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> > > > -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> > > > -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> > > > The problem exists, even after we only assign 32 vcpus to the linux guest.
> > > > Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> > > > The output of qemu and kernel is in attachments. Do you have any idea
> > > > about the problem and how to solve it?    
> > > 
> > > IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> > > vcpus?
> > > 
> > > Btw, could you please try adding this to the QEMU cmdline when testing
> > > with 32 vcpus:
> > > 
> > >   -global ioapic.version=0x20
> > > 
> > > I see that you were running RHEL 7.2 guest with a default e1000. In
> > > that case, we may need to boost ioapic version to 0x20.
> > > 
> > > Thanks,
> > > 
> > > -- peterx  
> > 
> > Peter,
> > 
> > Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> > 
> > ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> > 
> > 
> > [    0.350669] smpboot: Max logical packages: 9
> > [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> > [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > [    0.354627] DMAR: Host address width 39
> > [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> > [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> > [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> > [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> > [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> > [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> > [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.369373] PGD 0 
> > [    0.370258] Oops: 0002 [#1] SMP
> > [    0.371140] Modules linked in:
> > [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> > [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> > [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> > [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> > [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> > [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> > [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> > [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> > [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> > [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> > [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> > [    0.392114] Stack:
> > [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> > [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> > [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> > [    0.399695] Call Trace:
> > [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> > [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> > [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> > [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> > [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> > [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> > [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> > [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.418516]  RSP <ffff880039adbe28>
> > [    0.419461] CR2: 0000000000000000
> > [    0.420386] ---[ end trace f68728a0d3053b52 ]---  
> 
> Hi, Igor,
> 
> Thanks for the feedback!
> 
> Not sure whether this patch can fix it (since they looks alike):
> 
>   https://lkml.org/lkml/2016/8/4/345

It really seems to be the issue fixed by my patch. Best way to know
is to try it or try an upstream commit before 6b2c28471.

> 
> CC Luiz.
> 
> Thanks,
> 
> -- peterx
> 

Re: [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Tue, 9 Aug 2016 21:35:04 +0800
Peter Xu <peterx@redhat.com> wrote:

> On Tue, Aug 09, 2016 at 10:28:41AM +0200, Igor Mammedov wrote:
> > On Mon, 8 Aug 2016 16:57:14 +0800
> > Peter Xu <peterx@redhat.com> wrote:
> >   
> > > On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:  
> > > > HI, everyone.
> > > > 
> > > > We have done some tests after merging this patch set into the lastest qemu
> > > > master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> > > > some problems we have met.
> > > > 
> > > > 1. We can't boot up a 288 vcpus linux guest with CLI:
> > > > qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> > > > -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> > > > -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> > > > The problem exists, even after we only assign 32 vcpus to the linux guest.
> > > > Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> > > > The output of qemu and kernel is in attachments. Do you have any idea
> > > > about the problem and how to solve it?    
> > > 
> > > IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> > > vcpus?
> > > 
> > > Btw, could you please try adding this to the QEMU cmdline when testing
> > > with 32 vcpus:
> > > 
> > >   -global ioapic.version=0x20
> > > 
> > > I see that you were running RHEL 7.2 guest with a default e1000. In
> > > that case, we may need to boost ioapic version to 0x20.
> > > 
> > > Thanks,
> > > 
> > > -- peterx  
> > 
> > Peter,
> > 
> > Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> > 
> > ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> > 
> > 
> > [    0.350669] smpboot: Max logical packages: 9
> > [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> > [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > [    0.354627] DMAR: Host address width 39
> > [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> > [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> > [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> > [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> > [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> > [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> > [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.369373] PGD 0 
> > [    0.370258] Oops: 0002 [#1] SMP
> > [    0.371140] Modules linked in:
> > [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> > [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> > [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> > [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> > [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> > [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> > [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> > [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> > [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> > [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> > [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> > [    0.392114] Stack:
> > [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> > [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> > [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> > [    0.399695] Call Trace:
> > [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> > [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> > [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> > [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> > [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> > [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> > [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> > [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.418516]  RSP <ffff880039adbe28>
> > [    0.419461] CR2: 0000000000000000
> > [    0.420386] ---[ end trace f68728a0d3053b52 ]---  
> 
> Hi, Igor,
> 
> Thanks for the feedback!
> 
> Not sure whether this patch can fix it (since they looks alike):
> 
>   https://lkml.org/lkml/2016/8/4/345
it fixes this crash with upstream kernel,
however when it boots it still disables x2apic:

[    0.307554] smpboot: Max logical packages: 9
[    0.308529] smpboot: APIC(0) Converting physical 0 to logical package 0
[    0.309756] smpboot: APIC(11e) Converting physical 8 to logical package 1
[    0.311173] x2apic: IRQ remapping doesn't support X2APIC mode
[    0.312807] x2apic disabled
[    0.313935] Switched APIC routing to physical flat.

adding x2apic_phys to kernel's command line makes it crash but at another place:

[    0.364909] smpboot: Max logical packages: 9
[    0.365838] smpboot: APIC(0) Converting physical 0 to logical package 0
[    0.367183] smpboot: APIC(11e) Converting physical 8 to logical package 1
[    0.370291] x2apic: IRQ remapping doesn't support X2APIC mode
[    0.371901] x2apic disabled
[    0.376142] unchecked MSR access error: RDMSR from 0x802 at rIP: 0xffffffff81064787 (native_read_msr+0x7/0x40)
[    0.378511]  000000000000a110 ffff880008a0be30 ffffffff8105a33f ffff880008a0be40
[    0.380969]  ffffffff81051086 ffff880008a0be78 ffffffff81dafed7 ffffffff81f28640
[    0.383365]  ffff880008a00a58 ffffffff81c959c0 ffffffff818232c0 ffff880008a00000
[    0.385686] Call Trace:
[    0.386594]  [<ffffffff8105a33f>] native_apic_msr_read+0x1f/0x30
[    0.387985]  [<ffffffff81051086>] read_apic_id+0x16/0x30
[    0.389264]  [<ffffffff81dafed7>] native_smp_prepare_cpus+0x228/0x2d2
[    0.390648]  [<ffffffff81d9a28f>] kernel_init_freeable+0xd8/0x249
[    0.392038]  [<ffffffff816c1f3e>] kernel_init+0xe/0x110
[    0.393252]  [<ffffffff816ceeff>] ret_from_fork+0x1f/0x40
[    0.394504]  [<ffffffff816c1f30>] ? rest_init+0x80/0x80
[    0.395753] unchecked MSR access error: WRMSR to 0x808 (tried to write 0x0000000000000000) at rIP: 0xffffffff81064864 (native_write_msr+0x4/0x30)
[    0.398483]  ffffffff8105a310 ffff880008a0be28 ffffffff810542d9 0000000a95136142
[    0.400788]  000000000024940a 0000000000000000 0000000000000000 0000000000002000
[    0.403139]  000000000000a118 000000000000011f 0000000000000120 ffff880008a0be40
[    0.405406] Call Trace:
[    0.406304]  [<ffffffff8105a310>] ? native_apic_msr_write+0x30/0x40
[    0.407684]  [<ffffffff810542d9>] setup_local_APIC+0xb9/0x370
[    0.409048]  [<ffffffff81db22fc>] apic_bsp_setup+0x59/0xac
[    0.410305]  [<ffffffff81dafeff>] native_smp_prepare_cpus+0x250/0x2d2
[    0.411686]  [<ffffffff81d9a28f>] kernel_init_freeable+0xd8/0x249
[    0.413061]  [<ffffffff816c1f3e>] kernel_init+0xe/0x110
[    0.414252]  [<ffffffff816ceeff>] ret_from_fork+0x1f/0x40
[    0.415473]  [<ffffffff816c1f30>] ? rest_init+0x80/0x80


> 
> CC Luiz.
> 
> Thanks,
> 
> -- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Tue, 9 Aug 2016 21:35:04 +0800
Peter Xu <peterx@redhat.com> wrote:

> On Tue, Aug 09, 2016 at 10:28:41AM +0200, Igor Mammedov wrote:
> > On Mon, 8 Aug 2016 16:57:14 +0800
> > Peter Xu <peterx@redhat.com> wrote:
> >   
> > > On Mon, Aug 08, 2016 at 03:41:23PM +0800, Chao Gao wrote:  
> > > > HI, everyone.
> > > > 
> > > > We have done some tests after merging this patch set into the lastest qemu
> > > > master. In kvm aspect, we use the lastest kvm linux-next branch. Here are
> > > > some problems we have met.
> > > > 
> > > > 1. We can't boot up a 288 vcpus linux guest with CLI:
> > > > qemu-system-x86_64 -boot c -m 4096 -sdl -monitor pty --enable-kvm \
> > > > -M kernel-irqchip=split -serial stdio -bios bios.bin -smp cpus=288 \
> > > > -hda vdisk.img -device intel-iommu,intremap=on -machine q35.
> > > > The problem exists, even after we only assign 32 vcpus to the linux guest.
> > > > Maybe the output "do_IRQ: 146.113 No irq handler for vector (irq -1)" is a clue.
> > > > The output of qemu and kernel is in attachments. Do you have any idea
> > > > about the problem and how to solve it?    
> > > 
> > > IIUC, we need to wait for Radim's QEMU patches to finally enable 288
> > > vcpus?
> > > 
> > > Btw, could you please try adding this to the QEMU cmdline when testing
> > > with 32 vcpus:
> > > 
> > >   -global ioapic.version=0x20
> > > 
> > > I see that you were running RHEL 7.2 guest with a default e1000. In
> > > that case, we may need to boost ioapic version to 0x20.
> > > 
> > > Thanks,
> > > 
> > > -- peterx  
> > 
> > Peter,
> > 
> > Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> > 
> > ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> > 
> > 
> > [    0.350669] smpboot: Max logical packages: 9
> > [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> > [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > [    0.354627] DMAR: Host address width 39
> > [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> > [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> > [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> > [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> > [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> > [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> > [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.369373] PGD 0 
> > [    0.370258] Oops: 0002 [#1] SMP
> > [    0.371140] Modules linked in:
> > [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> > [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> > [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> > [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> > [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> > [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> > [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> > [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> > [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> > [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> > [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> > [    0.392114] Stack:
> > [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> > [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> > [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> > [    0.399695] Call Trace:
> > [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> > [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> > [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> > [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> > [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> > [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> > [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> > [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > [    0.418516]  RSP <ffff880039adbe28>
> > [    0.419461] CR2: 0000000000000000
> > [    0.420386] ---[ end trace f68728a0d3053b52 ]---  
> 
> Hi, Igor,
> 
> Thanks for the feedback!
> 
> Not sure whether this patch can fix it (since they looks alike):
> 
>   https://lkml.org/lkml/2016/8/4/345
it fixes this crash with upstream kernel,
however when it boots it still disables x2apic:

[    0.307554] smpboot: Max logical packages: 9
[    0.308529] smpboot: APIC(0) Converting physical 0 to logical package 0
[    0.309756] smpboot: APIC(11e) Converting physical 8 to logical package 1
[    0.311173] x2apic: IRQ remapping doesn't support X2APIC mode
[    0.312807] x2apic disabled
[    0.313935] Switched APIC routing to physical flat.

adding x2apic_phys to kernel's command line makes it crash but at another place:

[    0.364909] smpboot: Max logical packages: 9
[    0.365838] smpboot: APIC(0) Converting physical 0 to logical package 0
[    0.367183] smpboot: APIC(11e) Converting physical 8 to logical package 1
[    0.370291] x2apic: IRQ remapping doesn't support X2APIC mode
[    0.371901] x2apic disabled
[    0.376142] unchecked MSR access error: RDMSR from 0x802 at rIP: 0xffffffff81064787 (native_read_msr+0x7/0x40)
[    0.378511]  000000000000a110 ffff880008a0be30 ffffffff8105a33f ffff880008a0be40
[    0.380969]  ffffffff81051086 ffff880008a0be78 ffffffff81dafed7 ffffffff81f28640
[    0.383365]  ffff880008a00a58 ffffffff81c959c0 ffffffff818232c0 ffff880008a00000
[    0.385686] Call Trace:
[    0.386594]  [<ffffffff8105a33f>] native_apic_msr_read+0x1f/0x30
[    0.387985]  [<ffffffff81051086>] read_apic_id+0x16/0x30
[    0.389264]  [<ffffffff81dafed7>] native_smp_prepare_cpus+0x228/0x2d2
[    0.390648]  [<ffffffff81d9a28f>] kernel_init_freeable+0xd8/0x249
[    0.392038]  [<ffffffff816c1f3e>] kernel_init+0xe/0x110
[    0.393252]  [<ffffffff816ceeff>] ret_from_fork+0x1f/0x40
[    0.394504]  [<ffffffff816c1f30>] ? rest_init+0x80/0x80
[    0.395753] unchecked MSR access error: WRMSR to 0x808 (tried to write 0x0000000000000000) at rIP: 0xffffffff81064864 (native_write_msr+0x4/0x30)
[    0.398483]  ffffffff8105a310 ffff880008a0be28 ffffffff810542d9 0000000a95136142
[    0.400788]  000000000024940a 0000000000000000 0000000000000000 0000000000002000
[    0.403139]  000000000000a118 000000000000011f 0000000000000120 ffff880008a0be40
[    0.405406] Call Trace:
[    0.406304]  [<ffffffff8105a310>] ? native_apic_msr_write+0x30/0x40
[    0.407684]  [<ffffffff810542d9>] setup_local_APIC+0xb9/0x370
[    0.409048]  [<ffffffff81db22fc>] apic_bsp_setup+0x59/0xac
[    0.410305]  [<ffffffff81dafeff>] native_smp_prepare_cpus+0x250/0x2d2
[    0.411686]  [<ffffffff81d9a28f>] kernel_init_freeable+0xd8/0x249
[    0.413061]  [<ffffffff816c1f3e>] kernel_init+0xe/0x110
[    0.414252]  [<ffffffff816ceeff>] ret_from_fork+0x1f/0x40
[    0.415473]  [<ffffffff816c1f30>] ? rest_init+0x80/0x80


> 
> CC Luiz.
> 
> Thanks,
> 
> -- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Wed, Aug 10, 2016 at 10:51:51AM +0200, Igor Mammedov wrote:

[...]

> > > Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> > > 
> > > ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> > > 
> > > 
> > > [    0.350669] smpboot: Max logical packages: 9
> > > [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> > > [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > > [    0.354627] DMAR: Host address width 39
> > > [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> > > [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> > > [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> > > [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> > > [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> > > [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> > > [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > [    0.369373] PGD 0 
> > > [    0.370258] Oops: 0002 [#1] SMP
> > > [    0.371140] Modules linked in:
> > > [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> > > [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> > > [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> > > [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> > > [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> > > [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> > > [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> > > [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> > > [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> > > [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> > > [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> > > [    0.392114] Stack:
> > > [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> > > [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> > > [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> > > [    0.399695] Call Trace:
> > > [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> > > [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> > > [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> > > [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> > > [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> > > [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> > > [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> > > [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > [    0.418516]  RSP <ffff880039adbe28>
> > > [    0.419461] CR2: 0000000000000000
> > > [    0.420386] ---[ end trace f68728a0d3053b52 ]---  

I failed to reproduce this panic on my machine with parameter:

bin=x86_64-softmmu/qemu-system-x86_64
$bin -M q35,kernel-irqchip=split -enable-kvm -m 2048 \
     -monitor stdio -smp 4 \
     -device intel-iommu,intremap=on \
     -netdev user,id=net0,hostfwd=tcp::5555-:22 \
     -device e1000,netdev=net0 \
     -kernel /root/git/linux/arch/x86/boot/bzImage \
     -append root=/dev/sda3 \
     /root/images/rhel-7.2.qcow2

Guest kernel version is exactly 4.7.0+ (d52bd54db). In the guest, I
see x2apic enabled. Did I miss anything special?

[...]

> adding x2apic_phys to kernel's command line makes it crash but at another place:
> 
> [    0.364909] smpboot: Max logical packages: 9
> [    0.365838] smpboot: APIC(0) Converting physical 0 to logical package 0
> [    0.367183] smpboot: APIC(11e) Converting physical 8 to logical package 1
> [    0.370291] x2apic: IRQ remapping doesn't support X2APIC mode
> [    0.371901] x2apic disabled

Failed to understand why x2apic_phys will affect the system if x2apic
is disabled after all.

Thanks!

-- peterx

Re: [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Thu, 11 Aug 2016 13:10:57 +0800
Peter Xu <peterx@redhat.com> wrote:

> On Wed, Aug 10, 2016 at 10:51:51AM +0200, Igor Mammedov wrote:
> 
> [...]
> 
> > > > Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> > > > 
> > > > ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> > > > 
> > > > 
> > > > [    0.350669] smpboot: Max logical packages: 9
> > > > [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> > > > [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > > > [    0.354627] DMAR: Host address width 39
> > > > [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> > > > [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> > > > [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> > > > [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> > > > [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> > > > [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> > > > [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > > [    0.369373] PGD 0 
> > > > [    0.370258] Oops: 0002 [#1] SMP
> > > > [    0.371140] Modules linked in:
> > > > [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> > > > [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> > > > [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> > > > [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > > [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> > > > [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> > > > [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> > > > [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> > > > [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> > > > [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> > > > [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> > > > [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > > [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> > > > [    0.392114] Stack:
> > > > [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> > > > [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> > > > [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> > > > [    0.399695] Call Trace:
> > > > [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> > > > [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> > > > [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> > > > [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> > > > [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> > > > [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> > > > [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> > > > [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > > [    0.418516]  RSP <ffff880039adbe28>
> > > > [    0.419461] CR2: 0000000000000000
> > > > [    0.420386] ---[ end trace f68728a0d3053b52 ]---    
> 
> I failed to reproduce this panic on my machine with parameter:
> 
> bin=x86_64-softmmu/qemu-system-x86_64
> $bin -M q35,kernel-irqchip=split -enable-kvm -m 2048 \
>      -monitor stdio -smp 4 \
>      -device intel-iommu,intremap=on \
>      -netdev user,id=net0,hostfwd=tcp::5555-:22 \
>      -device e1000,netdev=net0 \
>      -kernel /root/git/linux/arch/x86/boot/bzImage \
>      -append root=/dev/sda3 \
>      /root/images/rhel-7.2.qcow2
> 
> Guest kernel version is exactly 4.7.0+ (d52bd54db). In the guest, I
> see x2apic enabled. Did I miss anything special?
> 
you missed presence of x2apic CPU which this series enables,
add/change CLI as following:

-smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
-device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0

+ add x2apic_phys to kernel's command line

PS:
the last kernel I've tried is:
   v4.8-rc1-53-ga0cba21 + fix from Luiz

> [...]
> 
> > adding x2apic_phys to kernel's command line makes it crash but at another place:
> > 
> > [    0.364909] smpboot: Max logical packages: 9
> > [    0.365838] smpboot: APIC(0) Converting physical 0 to logical package 0
> > [    0.367183] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > [    0.370291] x2apic: IRQ remapping doesn't support X2APIC mode
> > [    0.371901] x2apic disabled  
> 
> Failed to understand why x2apic_phys will affect the system if x2apic
> is disabled after all.
it looks like despite printing "x2apic disabled" is still tries to access
MSRs available only when CPU is in x2apic mode.

> 
> Thanks!
> 
> -- peterx
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Igor Mammedov]

On Thu, 11 Aug 2016 13:10:57 +0800
Peter Xu <peterx@redhat.com> wrote:

> On Wed, Aug 10, 2016 at 10:51:51AM +0200, Igor Mammedov wrote:
> 
> [...]
> 
> > > > Upstream guest kernel 4.7.0+ (d52bd54db) crashes when booting with irq remapping on:
> > > > 
> > > > ./qemu-system-x86_64 -enable-kvm -smp 1,sockets=9,cores=32,threads=1,maxcpus=288 -device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0 -bios x2apic_bios.bin  -m 1G -nographic -device intel-iommu,intremap=on -machine q35,kernel-irqchip=split -snapshot -global ioapic.version=0x20 /dev/rhel72
> > > > 
> > > > 
> > > > [    0.350669] smpboot: Max logical packages: 9
> > > > [    0.351853] smpboot: APIC(0) Converting physical 0 to logical package 0
> > > > [    0.353160] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > > > [    0.354627] DMAR: Host address width 39
> > > > [    0.355621] DMAR: DRHD base: 0x000000fed90000 flags: 0x1
> > > > [    0.356785] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 12008c22260206 ecap f00f1a
> > > > [    0.358721] DMAR-IR: IOAPIC id 0 under DRHD base  0xfed90000 IOMMU 0
> > > > [    0.360029] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
> > > > [    0.364605] DMAR-IR: Enabled IRQ remapping in x2apic mode
> > > > [    0.365805] BUG: unable to handle kernel NULL pointer dereference at           (null)
> > > > [    0.367965] IP: [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > > [    0.369373] PGD 0 
> > > > [    0.370258] Oops: 0002 [#1] SMP
> > > > [    0.371140] Modules linked in:
> > > > [    0.372150] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.7.0+ #647
> > > > [    0.373485] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.9.0-143-gbac87e4 04/01/2014
> > > > [    0.375622] task: ffff880039ad0000 task.stack: ffff880039ad8000
> > > > [    0.376875] RIP: 0010:[<ffffffff8105b025>]  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > > [    0.379066] RSP: 0000:ffff880039adbe28  EFLAGS: 00010202
> > > > [    0.380299] RAX: 0000000000000000 RBX: ffffffff81f388a8 RCX: ffff880039e00000
> > > > [    0.381677] RDX: 0000000000000000 RSI: 0000000000000002 RDI: 0000000000000246
> > > > [    0.383096] RBP: ffff880039adbe28 R08: 00000000000000c6 R09: ffff8800000b9b80
> > > > [    0.384579] R10: 00000000000000a0 R11: 0000000000000050 R12: 0000000000002000
> > > > [    0.385990] R13: 000000000000a118 R14: 000000000000011f R15: 0000000000000120
> > > > [    0.387448] FS:  0000000000000000(0000) GS:ffff880039e00000(0000) knlGS:0000000000000000
> > > > [    0.389454] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > > > [    0.390697] CR2: 0000000000000000 CR3: 0000000001c06000 CR4: 00000000000006f0
> > > > [    0.392114] Stack:
> > > > [    0.392889]  ffff880039adbe40 ffffffff81da277c 000000000000a110 ffff880039adbe78
> > > > [    0.395135]  ffffffff81d9c055 ffffffff81f14c60 ffff880039ad0a58 ffffffff81c95ac0
> > > > [    0.397469]  ffffffff818232c0 ffff880039ad0000 ffff880039adbf38 ffffffff81d86293
> > > > [    0.399695] Call Trace:
> > > > [    0.400529]  [<ffffffff81da277c>] default_setup_apic_routing+0x28/0x69
> > > > [    0.401881]  [<ffffffff81d9c055>] native_smp_prepare_cpus+0x223/0x2d2
> > > > [    0.403260]  [<ffffffff81d86293>] kernel_init_freeable+0xd8/0x249
> > > > [    0.404525]  [<ffffffff816d1b2e>] kernel_init+0xe/0x110
> > > > [    0.405703]  [<ffffffff816deb3f>] ret_from_fork+0x1f/0x40
> > > > [    0.406966]  [<ffffffff816d1b20>] ? rest_init+0x80/0x80
> > > > [    0.408165] Code: 00 31 c0 65 8b 15 2c f1 fa 7e 85 c9 75 01 c3 48 63 ca 55 48 c7 c0 10 d7 00 00 48 8b 0c cd 20 8d d4 81 89 d2 48 89 e5 48 8b 04 08 <f0> 48 0f ab 10 49 c7 c0 60 b0 05 81 48 c7 c1 a0 ae 05 81 ba 01 
> > > > [    0.417107] RIP  [<ffffffff8105b025>] x2apic_cluster_probe+0x35/0x70
> > > > [    0.418516]  RSP <ffff880039adbe28>
> > > > [    0.419461] CR2: 0000000000000000
> > > > [    0.420386] ---[ end trace f68728a0d3053b52 ]---    
> 
> I failed to reproduce this panic on my machine with parameter:
> 
> bin=x86_64-softmmu/qemu-system-x86_64
> $bin -M q35,kernel-irqchip=split -enable-kvm -m 2048 \
>      -monitor stdio -smp 4 \
>      -device intel-iommu,intremap=on \
>      -netdev user,id=net0,hostfwd=tcp::5555-:22 \
>      -device e1000,netdev=net0 \
>      -kernel /root/git/linux/arch/x86/boot/bzImage \
>      -append root=/dev/sda3 \
>      /root/images/rhel-7.2.qcow2
> 
> Guest kernel version is exactly 4.7.0+ (d52bd54db). In the guest, I
> see x2apic enabled. Did I miss anything special?
> 
you missed presence of x2apic CPU which this series enables,
add/change CLI as following:

-smp 1,sockets=9,cores=32,threads=1,maxcpus=288 \
-device qemu64-x86_64-cpu,socket-id=8,core-id=30,thread-id=0

+ add x2apic_phys to kernel's command line

PS:
the last kernel I've tried is:
   v4.8-rc1-53-ga0cba21 + fix from Luiz

> [...]
> 
> > adding x2apic_phys to kernel's command line makes it crash but at another place:
> > 
> > [    0.364909] smpboot: Max logical packages: 9
> > [    0.365838] smpboot: APIC(0) Converting physical 0 to logical package 0
> > [    0.367183] smpboot: APIC(11e) Converting physical 8 to logical package 1
> > [    0.370291] x2apic: IRQ remapping doesn't support X2APIC mode
> > [    0.371901] x2apic disabled  
> 
> Failed to understand why x2apic_phys will affect the system if x2apic
> is disabled after all.
it looks like despite printing "x2apic disabled" is still tries to access
MSRs available only when CPU is in x2apic mode.

> 
> Thanks!
> 
> -- peterx
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Chao Gao]

Hi, we had 3 problems left here.
1. IRQremapping can't work with x2apic_cluster mode.
2. apic_id > 255 can't receive devices interrupts.
3. windows crash when present IRQremapping capability to it.

I test with latest kernel v4.8-rc7 and find all of them are unsolved.
Also in the qemu and kernel code bases, I couldn't find some patches to 
solve them. Will you fix these problems or leave them to communities? 

Thanks,
-Chao

On Tue, Aug 09, 2016 at 02:51:16PM +0200, Radim Krčmář wrote:
>2016-08-09 16:19+0800, Chao Gao:
>> On Tue, Aug 09, 2016 at 02:18:15PM +0800, Peter Xu wrote:
>>>I think the problem is with x2apic. Currently, x2apic is enabled in
>>>vIOMMU when kernel irqchip is used. This is problematic, since
>>>actually we are throughing away dest_id[31:8] without Radim's patches,
>>>meanwhile I see that by default x2apic is using cluster mode.
>>>
>>>In cluster mode, 8 bits will possibly not suffice (I think the reason
>>>why >17 vcpus will bring trouble is that each cluster has 16 vcpus,
>>>we'll have trouble if we have more than one cluster).
>>>
>>>To temporarily solve your issue, you should not only need "-global
>>>ioapic.version=0x20" in QEMU command line, but also add "x2apic_phys"
>>>to you guest kernel boot parameter, to force guest boot with x2apic
>>>physical mode (not cluster mode). Though this can only work for <255
>>>vcpus. IMHO we may still need to wait for Radim's patches to test >255
>>>case.
>> 
>> ok, after adding "x2apic_phys" to guest kernel boot parameter, I 
>> boot up a 288(yes, 288) vcpus guest successfully with command
>> qemu-system-x86_64 -boot c -m 4096 -sdl --enable-kvm \
>> -M kernel-irqchip=split -bios bios.bin -smp cpus=288 -hda vdisk.img \
>> -device intel-iommu,intremap=on -machine q35
>> Also, I can see interrupts on those cpu with inital apicid>255 from /proc/cpuinfo and /proc/interrupts. 
>
>Great, thanks for testing!
>Only IPIs will be correctly delivered to apic_id > 255 without few more
>patches on the QEMU side, though.
>

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Peter Xu]

On Thu, Sep 22, 2016 at 12:34:36PM +0800, Chao Gao wrote:
> Hi, we had 3 problems left here.
> 1. IRQremapping can't work with x2apic_cluster mode.
> 2. apic_id > 255 can't receive devices interrupts.
> 3. windows crash when present IRQremapping capability to it.

For (3), I don't know whether it's urgent or not - I've put it into my
todo list (assuming this is for HPC and mostly we are using Linux
guests?).

For (1-2), again we may need to wait for Radim's patches.

Thanks.

-- peterx

Re: [Qemu-devel] [PATCH for-2.8 00/18] pc: q35: x2APIC support in kvm_apic mode

[Lan Tianyu]

On 2016年09月23日 13:26, Peter Xu wrote:
> On Thu, Sep 22, 2016 at 12:34:36PM +0800, Chao Gao wrote:
>> Hi, we had 3 problems left here.
>> 1. IRQremapping can't work with x2apic_cluster mode.
>> 2. apic_id > 255 can't receive devices interrupts.
>> 3. windows crash when present IRQremapping capability to it.
> 

Hi Peter:

Thanks for your information. We just want to make sure these issues
aren't ignored.

> For (3), I don't know whether it's urgent or not - I've put it into my
> todo list (assuming this is for HPC and mostly we are using Linux
> guests?).
> 
> For (1-2), again we may need to wait for Radim's patches.