* PVH Dom0 Intel IOMMU issues
@ 2017-04-14 15:34 Roger Pau Monné
2017-04-16 23:32 ` Chao Gao
2017-04-18 3:04 ` Tian, Kevin
0 siblings, 2 replies; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-14 15:34 UTC (permalink / raw)
To: xen-devel; +Cc: Kevin Tian
Hello,
Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
I'm not able to debug that in any meaningful way because the box seems to lock
up completely, even the watchdog NMI stops working. Here is the boot log, up to
the point where it freezes:
(XEN) Xen version 4.9-unstable (root@) (FreeBSD clang version 3.9.0 (tags/RELEASE_390/final 280324) (based on LLVM 3.9.0)) debug=y Fri Apr 14 16:15:35 BST 2017
(XEN) Latest ChangeSet:
(XEN) Console output is synchronous.
(XEN) Bootloader: FreeBSD Loader
(XEN) Command line: dom0_mem=4096M dom0=pvh com1=115200,8n1 console=com1,vga guest_loglvl=all loglvl=all iommu=debug,verbose sync_console
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 2 MBR signatures
(XEN) Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000008dc00 (usable)
(XEN) 000000000008dc00 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 0000000018ebb000 (usable)
(XEN) 0000000018ebb000 - 0000000018fe8000 (ACPI NVS)
(XEN) 0000000018fe8000 - 0000000018fe9000 (usable)
(XEN) 0000000018fe9000 - 0000000019000000 (ACPI NVS)
(XEN) 0000000019000000 - 000000001dffd000 (usable)
(XEN) 000000001dffd000 - 000000001e000000 (ACPI data)
(XEN) 000000001e000000 - 00000000ac784000 (usable)
(XEN) 00000000ac784000 - 00000000ac818000 (reserved)
(XEN) 00000000ac818000 - 00000000ad800000 (usable)
(XEN) 00000000b0000000 - 00000000b4000000 (reserved)
(XEN) 00000000fed20000 - 00000000fed40000 (reserved)
(XEN) 00000000fed50000 - 00000000fed90000 (reserved)
(XEN) 00000000ffa00000 - 00000000ffa40000 (reserved)
(XEN) 0000000100000000 - 0000000250000000 (usable)
(XEN) New Xen image base address: 0xad200000
(XEN) ACPI: RSDP 000FE300, 0024 (r2 DELL )
(XEN) ACPI: XSDT 1DFFEE18, 0074 (r1 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: FACP 18FEFD98, 00F4 (r4 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: DSDT 18FA9018, 6373 (r1 DELL CBX3 0 INTL 20091112)
(XEN) ACPI: FACS 18FF1F40, 0040
(XEN) ACPI: APIC 1DFFDC18, 0158 (r2 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: MCFG 18FFED18, 003C (r1 A M I OEMMCFG. 6222004 MSFT 97)
(XEN) ACPI: TCPA 18FFEC98, 0032 (r2 0 0)
(XEN) ACPI: SSDT 18FF0A98, 0306 (r1 DELLTP TPM 3000 INTL 20091112)
(XEN) ACPI: HPET 18FFEC18, 0038 (r1 A M I PCHHPET 6222004 AMI. 3)
(XEN) ACPI: BOOT 18FFEB98, 0028 (r1 DELL CBX3 6222004 AMI 10013)
(XEN) ACPI: SSDT 18FB0018, 36FFE (r2 INTEL CpuPm 4000 INTL 20091112)
(XEN) ACPI: SLIC 18FEEC18, 0176 (r3 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: DMAR 18FF1B18, 0094 (r1 A M I OEMDMAR 1 INTL 1)
(XEN) System RAM: 8149MB (8345288kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-0000000250000000
(XEN) Domain heap initialised
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 45 (0x2d), Stepping 7 (raw 000206d7)
(XEN) found SMP MP-table at 000f1db0
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408 (32 bits)
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 18ffdf40/0000000018ff1f40, using 32
(XEN) ACPI: wakeup_vec[18ffdf4c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x05] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x08] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x09] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0a] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0b] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0c] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0d] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0e] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x0f] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x10] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x11] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x12] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x13] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x14] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x15] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x16] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x17] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x19] lapic_id[0x18] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x19] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1a] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1b] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x1c] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x1d] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x1e] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x20] lapic_id[0x1f] disabled)
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec3f000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec3f000, GSI 24-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]Host address width 46
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D] dmaru->address = fbffe000
(XEN) [VT-D]drhd->address = fbffe000 iommu->reg = ffff82c00021b000
(XEN) [VT-D]cap = d2078c106f0462 ecap = f020fa
(XEN) [VT-D] IOAPIC: 0000:00:1f.7
(XEN) [VT-D] IOAPIC: 0000:00:05.4
(XEN) [VT-D] MSI HPET: 0000:f0:0f.0
(XEN) [VT-D] flags: INCLUDE_ALL
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:1d.0
(XEN) [VT-D] endpoint: 0000:00:1a.0
(XEN) [VT-D]dmar.c:638: RMRR region: base_addr ac7cf000 end_addr ac7defff
(XEN) [VT-D]found ACPI_DMAR_RHSA:
(XEN) [VT-D] rhsau->address: fbffe000 rhsau->proximity_domain: 0
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 32 CPUs (28 hotplug CPUs)
(XEN) IRQ limits: 48 GSI, 736 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) mce_intel.c:732: MCA capability: firstbank 0, 0 ext MSRs, BCAST, SER, CMCI
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 2992.809 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d08041d1f0 -> ffff82d08041d964
(XEN) PCI: MCFG configuration 0: base b0000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at b0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 9
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x21120
(XEN) mwait-idle: v0.4.1 model 0x2d
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 4 CPUs
(XEN) Running stub recovery selftests...
(XEN) traps.c:3466: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080346112
(XEN) traps.c:813: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] -> ffff82d080346112
(XEN) traps.c:1215: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080346112
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 624 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) ** Building a PVH Dom0 **
(XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map
(XEN) Found masked UR signaling on 0000:00:00.0
(XEN) Found masked UR signaling on 0000:00:01.0
(XEN) Found masked UR signaling on 0000:00:01.1
(XEN) Found masked UR signaling on 0000:00:02.0
(XEN) Found masked UR signaling on 0000:00:03.0
(XEN) [VT-D]d0:PCIe: map 0000:00:05.0
(XEN) Masked VT-d error signaling on 0000:00:05.0
(XEN) [VT-D]d0:PCIe: map 0000:00:05.2
(XEN) [VT-D]d0:PCI: map 0000:00:05.4
(XEN) [VT-D]d0:PCI: map 0000:00:16.0
(XEN) [VT-D]d0:PCI: map 0000:00:19.0
(XEN) [VT-D]d0:PCI: map 0000:00:1a.0
(XEN) [VT-D]d0:PCIe: map 0000:00:1b.0
(XEN) [VT-D]d0:PCI: map 0000:00:1d.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.2
(XEN) [VT-D]d0:PCI: map 0000:00:1f.3
(XEN) [VT-D]d0:PCIe: map 0000:03:00.0
(XEN) [VT-D]d0:PCIe: map 0000:03:00.1
(XEN) [VT-D]d0:PCIe: map 0000:05:00.0
(XEN) [VT-D]d0:PCIe: map 0000:05:00.3
(XEN) [VT-D]d0:PCIe: map 0000:07:00.0
(XEN) [VT-D]d0:PCI: map 0000:3f:08.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:08.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:08.4
(XEN) [VT-D]d0:PCI: map 0000:3f:09.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:09.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:09.4
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.1
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.2
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.3
(XEN) [VT-D]d0:PCI: map 0000:3f:0b.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0b.3
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.1
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.6
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.7
(XEN) [VT-D]d0:PCI: map 0000:3f:0d.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0d.1
(XEN) [VT-D]d0:PCI: map 0000:3f:0d.6
(XEN) [VT-D]d0:PCI: map 0000:3f:0e.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0e.1
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.1
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.2
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.4
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.5
(XEN) [VT-D]d0:PCI: map 0000:3f:0f.6
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.1
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.2
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.4
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.5
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.6
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.7
(XEN) [VT-D]d0:PCI: map 0000:3f:11.0
(XEN) [VT-D]d0:PCI: map 0000:3f:13.0
(XEN) [VT-D]d0:PCI: map 0000:3f:13.1
(XEN) [VT-D]d0:PCI: map 0000:3f:13.4
(XEN) [VT-D]d0:PCI: map 0000:3f:13.5
(XEN) [VT-D]d0:PCI: map 0000:3f:13.6
(XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021b000
You don't even need a PVHv2 capable Dom0 kernel in order to test this, just
adding "dom0=pvh" to the Xen command line should be enough to trigger this
issue. I'm really out of ideas about what to try next. The CPU model in the
above dmesg is a Xeon E5-1607.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-14 15:34 PVH Dom0 Intel IOMMU issues Roger Pau Monné
@ 2017-04-16 23:32 ` Chao Gao
2017-04-17 7:47 ` Roger Pau Monné
2017-04-18 3:04 ` Tian, Kevin
1 sibling, 1 reply; 16+ messages in thread
From: Chao Gao @ 2017-04-16 23:32 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
>Hello,
>
>Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
>different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
>completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
>OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
>
>I'm not able to debug that in any meaningful way because the box seems to lock
>up completely, even the watchdog NMI stops working. Here is the boot log, up to
>the point where it freezes:
I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
Xen 4.9-unstable
(XEN) [ 0.000000] Xen version 4.9-unstable (root@sh.intel.com) (gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-11)) debug=y Mon Apr 17 04:41:08 CST 2017
(XEN) [ 0.000000] Latest ChangeSet: Mon Apr 10 17:32:01 2017 +0200 git:17cd662
(XEN) [ 0.000000] Bootloader: GRUB 2.02~beta2
(XEN) [ 0.000000] Command line: conring_size=16m iommu=verbose,debug loglvl=all guest_loglvl=all com1=115200,8n1,0x3f8,4 console=com1,vga console_timestamps=boot vvtd_debug=0x3a dom0_mem=10G dom0=pvh
(XEN) [ 0.000000] Xen image load base address: 0
(XEN) [ 0.000000] Video information:
(XEN) [ 0.000000] VGA is text mode 80x25, font 8x16
(XEN) [ 0.000000] VBE/DDC methods: none; EDID transfer time: 1 seconds
(XEN) [ 0.000000] EDID info not retrieved because no DDC retrieval method detected
(XEN) [ 0.000000] Disc information:
(XEN) [ 0.000000] Found 1 MBR signatures
(XEN) [ 0.000000] Found 1 EDD information structures
(XEN) [ 0.000000] Xen-e820 RAM map:
(XEN) [ 0.000000] 0000000000000000 - 0000000000099800 (usable)
(XEN) [ 0.000000] 0000000000099800 - 00000000000a0000 (reserved)
(XEN) [ 0.000000] 00000000000e0000 - 0000000000100000 (reserved)
(XEN) [ 0.000000] 0000000000100000 - 0000000067b3b000 (usable)
(XEN) [ 0.000000] 0000000067b3b000 - 0000000067d62000 (reserved)
(XEN) [ 0.000000] 0000000067d62000 - 00000000681fc000 (usable)
(XEN) [ 0.000000] 00000000681fc000 - 000000006829f000 (ACPI data)
(XEN) [ 0.000000] 000000006829f000 - 000000006908a000 (usable)
(XEN) [ 0.000000] 000000006908a000 - 000000006a08a000 (reserved)
(XEN) [ 0.000000] 000000006a08a000 - 000000006b6e6000 (usable)
(XEN) [ 0.000000] 000000006b6e6000 - 000000006b9e6000 (reserved)
(XEN) [ 0.000000] 000000006b9e6000 - 000000006c416000 (ACPI NVS)
(XEN) [ 0.000000] 000000006c416000 - 000000006c516000 (ACPI data)
(XEN) [ 0.000000] 000000006c516000 - 000000006fb00000 (usable)
(XEN) [ 0.000000] 000000006fb00000 - 0000000090000000 (reserved)
(XEN) [ 0.000000] 00000000fd000000 - 00000000fe800000 (reserved)
(XEN) [ 0.000000] 00000000fec00000 - 00000000fec01000 (reserved)
(XEN) [ 0.000000] 00000000fec80000 - 00000000fed01000 (reserved)
(XEN) [ 0.000000] 00000000ff800000 - 0000000100c00000 (reserved)
(XEN) [ 0.000000] 0000000100c00000 - 0000001080000000 (usable)
(XEN) [ 0.000000] New Xen image base address: 0x6f400000
(XEN) [ 0.000000] ACPI: RSDP 000F0510, 0024 (r2 INTEL )
(XEN) [ 0.000000] ACPI: XSDT 6C42C188, 0104 (r1 INTEL S2600WF 0 INTL 20091013)
(XEN) [ 0.000000] ACPI: FACP 6C512000, 010C (r5 INTEL S2600WF 0 INTL 20091013)
(XEN) [ 0.000000] ACPI: DSDT 6C4B4000, 36756 (r2 INTEL S2600WF 3 INTL 20091013)
(XEN) [ 0.000000] ACPI: FACS 6C38E000, 0040
(XEN) [ 0.000000] ACPI: SSDT 6C513000, 04B0 (r2 INTEL S2600WF 0 MSFT 100000D)
(XEN) [ 0.000000] ACPI: UEFI 6C405000, 0042 (r1 INTEL S2600WF 2 INTL 20091013)
(XEN) [ 0.000000] ACPI: UEFI 6C390000, 005C (r1 INTEL RstUefiV 0 0)
(XEN) [ 0.000000] ACPI: HPET 6C511000, 0038 (r1 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: APIC 6C50F000, 16DE (r3 INTEL S2600WF 0 INTL 20091013)
(XEN) [ 0.000000] ACPI: MCFG 6C50E000, 003C (r1 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: MSCT 6C50D000, 0090 (r1 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: NFIT 6C4F4000, 18028 (r1 0 0)
(XEN) [ 0.000000] ACPI: PCAT 6C4F3000, 0048 (r1 INTEL S2600WF 2 INTL 20091013)
(XEN) [ 0.000000] ACPI: PCCT 6C4F2000, 00AC (r1 INTEL S2600WF 2 INTL 20091013)
(XEN) [ 0.000000] ACPI: RASF 6C4F1000, 0030 (r1 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: SLIT 6C4F0000, 006C (r1 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: SRAT 6C4ED000, 2830 (r3 INTEL S2600WF 2 INTL 20091013)
(XEN) [ 0.000000] ACPI: SPMI 6C4EC000, 0041 (r5 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: WDDT 6C4EB000, 0040 (r1 INTEL S2600WF 0 INTL 20091013)
(XEN) [ 0.000000] ACPI: OEM4 681FC000, A27C4 (r2 INTEL CPU CST 3000 INTL 20140828)
(XEN) [ 0.000000] ACPI: OEM1 6C489000, 2A2C4 (r2 INTEL CPU EIST 3000 INTL 20140828)
(XEN) [ 0.000000] ACPI: SSDT 6C455000, 33990 (r2 INTEL S2600WF 4000 INTL 20091013)
(XEN) [ 0.000000] ACPI: OEM3 6C42F000, 25F64 (r2 INTEL CPU TST 3000 INTL 20140828)
(XEN) [ 0.000000] ACPI: SSDT 6C42D000, 134E (r2 INTEL S2600WF 2 INTL 20091013)
(XEN) [ 0.000000] ACPI: SPCR 6C514000, 0050 (r1 INTEL S2600WF 0 INTL 20091013)
(XEN) [ 0.000000] ACPI: DMAR 6C42B000, 01C0 (r1 INTEL S2600WF 1 INTL 20091013)
(XEN) [ 0.000000] ACPI: HEST 6C42A000, 00A8 (r1 INTEL S2600WF 1 INTL 1)
(XEN) [ 0.000000] ACPI: BERT 6C429000, 0030 (r1 INTEL S2600WF 1 INTL 1)
(XEN) [ 0.000000] ACPI: ERST 6C428000, 0230 (r1 INTEL S2600WF 1 INTL 1)
(XEN) [ 0.000000] ACPI: EINJ 6C427000, 0150 (r1 INTEL S2600WF 1 INTL 1)
(XEN) [ 0.000000] ACPI: BGRT 6C426000, 0038 (r1 INTEL S2600WF 2 INTL 20091013)
(XEN) [ 0.000000] System RAM: 65229MB (66795132kB)
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 08 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 0a -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 0c -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 10 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 12 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 14 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 16 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 18 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 1a -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 1c -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 20 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 22 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 24 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 26 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 28 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 2a -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 2c -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 30 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 32 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 34 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 36 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 38 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 3a -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 3c -> Node 0
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 40 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 42 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 44 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 46 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 48 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 4a -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 4c -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 50 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 52 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 54 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 56 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 58 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 5a -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 5c -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 60 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 62 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 64 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 66 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 68 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 6a -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 6c -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 70 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 72 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 74 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 76 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 78 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 7a -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 7c -> Node 1
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 01 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 03 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 05 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 07 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 09 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 0b -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 0d -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 11 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 13 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 15 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 17 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 19 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 1b -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 1d -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 21 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 23 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 25 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 27 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 29 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 2b -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 2d -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 31 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 33 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 35 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 37 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 39 -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 3b -> Node 0
(XEN) [ 0.000000] SRAT: PXM 0 -> APIC 3d -> Node 0
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 41 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 43 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 45 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 47 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 49 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 4b -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 4d -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 51 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 53 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 55 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 57 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 59 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 5b -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 5d -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 61 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 63 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 65 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 67 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 69 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 6b -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 6d -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 71 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 73 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 75 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 77 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 79 -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 7b -> Node 1
(XEN) [ 0.000000] SRAT: PXM 1 -> APIC 7d -> Node 1
(XEN) [ 0.000000] SRAT: Node 0 PXM 0 0-80000000
(XEN) [ 0.000000] SRAT: Node 0 PXM 0 100000000-880000000
(XEN) [ 0.000000] SRAT: Node 1 PXM 1 880000000-1080000000
(XEN) [ 0.000000] NUMA: Using 19 for the hash shift.
(XEN) [ 0.000000] Domain heap initialised DMA width 32 bits
(XEN) [ 0.000000] Allocated console ring of 16384 KiB.
(XEN) [ 0.000000] CPU Vendor: Intel, Family 6 (0x6), Model 85 (0x55), Stepping 2 (raw 00050652)
(XEN) [ 0.000000] DMI 2.8 present.
(XEN) [ 0.000000] Using APIC driver default
(XEN) [ 0.000000] ACPI: PM-Timer IO Port: 0x508 (32 bits)
(XEN) [ 0.000000] ACPI: v5 SLEEP INFO: control[0:0], status[0:0]
(XEN) [ 0.000000] ACPI: SLEEP INFO: pm1x_cnt[1:504,1:0], pm1x_evt[1:500,1:0]
(XEN) [ 0.000000] ACPI: 32/64X FACS address mismatch in FADT - 6c38e000/0000000000000000, using 32
(XEN) [ 0.000000] ACPI: wakeup_vec[6c38e00c], vec_size[20]
(XEN) [ 0.000000] ACPI: Local APIC address 0xfee00000
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x04] lapic_id[0x04] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x06] lapic_id[0x06] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x08] lapic_id[0x08] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x0a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x10] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x10] lapic_id[0x12] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x12] lapic_id[0x14] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x14] lapic_id[0x16] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x16] lapic_id[0x18] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x18] lapic_id[0x1a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x1c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x20] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x22] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x20] lapic_id[0x24] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x22] lapic_id[0x26] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x24] lapic_id[0x28] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x26] lapic_id[0x2a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x28] lapic_id[0x2c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x2a] lapic_id[0x30] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x2c] lapic_id[0x32] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x2e] lapic_id[0x34] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x30] lapic_id[0x36] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x32] lapic_id[0x38] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x34] lapic_id[0x3a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x36] lapic_id[0x3c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x38] lapic_id[0x40] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x3a] lapic_id[0x42] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x3c] lapic_id[0x44] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x3e] lapic_id[0x46] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x40] lapic_id[0x48] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x42] lapic_id[0x4a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x44] lapic_id[0x4c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x46] lapic_id[0x50] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x48] lapic_id[0x52] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x4a] lapic_id[0x54] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x4c] lapic_id[0x56] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x4e] lapic_id[0x58] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x50] lapic_id[0x5a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x52] lapic_id[0x5c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x54] lapic_id[0x60] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x56] lapic_id[0x62] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x58] lapic_id[0x64] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x5a] lapic_id[0x66] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x5c] lapic_id[0x68] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x5e] lapic_id[0x6a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x60] lapic_id[0x6c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x62] lapic_id[0x70] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x64] lapic_id[0x72] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x66] lapic_id[0x74] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x68] lapic_id[0x76] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x6a] lapic_id[0x78] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x6c] lapic_id[0x7a] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x6e] lapic_id[0x7c] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x05] lapic_id[0x05] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x07] lapic_id[0x07] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x09] lapic_id[0x09] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x11] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x11] lapic_id[0x13] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x13] lapic_id[0x15] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x15] lapic_id[0x17] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x17] lapic_id[0x19] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x19] lapic_id[0x1b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x21] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x23] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x21] lapic_id[0x25] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x23] lapic_id[0x27] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x25] lapic_id[0x29] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x27] lapic_id[0x2b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x29] lapic_id[0x2d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x2b] lapic_id[0x31] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x2d] lapic_id[0x33] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x2f] lapic_id[0x35] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x31] lapic_id[0x37] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x33] lapic_id[0x39] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x35] lapic_id[0x3b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x37] lapic_id[0x3d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x39] lapic_id[0x41] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x3b] lapic_id[0x43] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x3d] lapic_id[0x45] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x3f] lapic_id[0x47] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x41] lapic_id[0x49] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x43] lapic_id[0x4b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x45] lapic_id[0x4d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x47] lapic_id[0x51] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x49] lapic_id[0x53] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x4b] lapic_id[0x55] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x4d] lapic_id[0x57] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x4f] lapic_id[0x59] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x51] lapic_id[0x5b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x53] lapic_id[0x5d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x55] lapic_id[0x61] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x57] lapic_id[0x63] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x59] lapic_id[0x65] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x5b] lapic_id[0x67] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x5d] lapic_id[0x69] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x5f] lapic_id[0x6b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x61] lapic_id[0x6d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x63] lapic_id[0x71] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x65] lapic_id[0x73] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x67] lapic_id[0x75] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x69] lapic_id[0x77] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x6b] lapic_id[0x79] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x6d] lapic_id[0x7b] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0x6f] lapic_id[0x7d] enabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: LAPIC (acpi_id[0xff] lapic_id[0xff] disabled)
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x00] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x01] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x02] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x03] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x04] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x05] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x06] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x07] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x08] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x09] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x0a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x0b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x0c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x0d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x0e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xe beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x0f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xf beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x10] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x10 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x11] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x11 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x12] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x12 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x13] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x13 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x14] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x14 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x15] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x15 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x16] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x16 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x17] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x17 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x18] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x18 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x19] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x19 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x1a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x1b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x1c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x1d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x1e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x1f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x1f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x20] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x20 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x21] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x21 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x22] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x22 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x23] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x23 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x24] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x24 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x25] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x25 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x26] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x26 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x27] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x27 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x28] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x28 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x29] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x29 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x2a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x2b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x2c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x2d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x2e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x2f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x2f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x30] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x30 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x31] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x31 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x32] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x32 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x33] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x33 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x34] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x34 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x35] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x35 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x36] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x36 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x37] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x37 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x38] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x38 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x39] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x39 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x3a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x3b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x3c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x3d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x3e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x3f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x3f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x40] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x40 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x41] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x41 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x42] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x42 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x43] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x43 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x44] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x44 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x45] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x45 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x46] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x46 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x47] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x47 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x48] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x48 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x49] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x49 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x4a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x4b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x4c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x4d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x4e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x4f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x4f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x50] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x50 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x51] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x51 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x52] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x52 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x53] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x53 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x54] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x54 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x55] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x55 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x56] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x56 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x57] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x57 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x58] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x58 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x59] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x59 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x5a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x5b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x5c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x5d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x5e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x5f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x5f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x60] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x60 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x61] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x61 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x62] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x62 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x63] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x63 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x64] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x64 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x65] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x65 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x66] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x66 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x67] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x67 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x68] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x68 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x69] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x69 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x6a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x6b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x6c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x6d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x6e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x6f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x6f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x70] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x70 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x71] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x71 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x72] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x72 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x73] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x73 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x74] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x74 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x75] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x75 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x76] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x76 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x77] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x77 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x78] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x78 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x79] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x79 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x7a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x7b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x7c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x7d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x7e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x7f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x7f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x80] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x80 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x81] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x81 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x82] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x82 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x83] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x83 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x84] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x84 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x85] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x85 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x86] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x86 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x87] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x87 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x88] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x88 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x89] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x89 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x8a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x8b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x8c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x8d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x8e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x8f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x8f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x90] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x90 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x91] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x91 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x92] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x92 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x93] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x93 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x94] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x94 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x95] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x95 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x96] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x96 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x97] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x97 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x98] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x98 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x99] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x99 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x9a] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9a beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x9b] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9b beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x9c] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9c beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x9d] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9d beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x9e] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9e beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0x9f] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0x9f beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa0] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa0 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa1] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa1 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa2] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa2 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa3] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa3 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa4] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa4 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa5] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa5 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa6] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa6 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa7] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa7 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa8] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa8 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xa9] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xa9 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xaa] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xaa beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xab] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xab beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xac] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xac beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xad] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xad beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xae] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xae beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xaf] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xaf beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb0] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb0 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb1] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb1 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb2] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb2 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb3] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb3 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb4] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb4 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb5] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb5 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb6] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb6 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb7] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb7 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb8] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb8 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xb9] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xb9 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xba] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xba beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xbb] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xbb beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xbc] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xbc beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xbd] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xbd beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xbe] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xbe beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xbf] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xbf beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc0] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc0 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc1] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc1 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc2] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc2 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc3] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc3 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc4] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc4 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc5] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc5 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc6] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc6 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc7] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc7 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc8] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc8 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xc9] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xc9 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xca] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xca beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xcb] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xcb beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xcc] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xcc beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xcd] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xcd beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xce] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xce beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xcf] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xcf beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd0] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd0 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd1] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd1 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd2] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd2 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd3] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd3 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd4] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd4 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd5] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd5 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd6] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd6 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd7] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd7 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd8] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd8 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xd9] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xd9 beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xda] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xda beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xdb] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xdb beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xdc] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xdc beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xdd] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xdd beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xde] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xde beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: X2APIC (apic_id[0xffffffff] uid[0xdf] disabled)
(XEN) [ 0.000000] APIC ID 0xffffffff and/or ACPI ID 0xdf beyond limit - processor ignored
(XEN) [ 0.000000] ACPI: LAPIC_NMI (acpi_id[0xff] high level lint[0x1])
(XEN) [ 0.000000] ACPI: X2APIC_NMI (uid[0xffffffff] high level lint[0x1])
(XEN) [ 0.000000] Overriding APIC driver with bigsmp
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
(XEN) [ 0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x09] address[0xfec01000] gsi_base[24])
(XEN) [ 0.000000] IOAPIC[1]: apic_id 9, version 32, address 0xfec01000, GSI 24-31
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x0a] address[0xfec08000] gsi_base[32])
(XEN) [ 0.000000] IOAPIC[2]: apic_id 10, version 32, address 0xfec08000, GSI 32-39
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x0b] address[0xfec10000] gsi_base[40])
(XEN) [ 0.000000] IOAPIC[3]: apic_id 11, version 32, address 0xfec10000, GSI 40-47
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x0c] address[0xfec18000] gsi_base[48])
(XEN) [ 0.000000] IOAPIC[4]: apic_id 12, version 32, address 0xfec18000, GSI 48-55
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x0f] address[0xfec20000] gsi_base[72])
(XEN) [ 0.000000] IOAPIC[5]: apic_id 15, version 32, address 0xfec20000, GSI 72-79
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x10] address[0xfec28000] gsi_base[80])
(XEN) [ 0.000000] IOAPIC[6]: apic_id 16, version 32, address 0xfec28000, GSI 80-87
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x11] address[0xfec30000] gsi_base[88])
(XEN) [ 0.000000] IOAPIC[7]: apic_id 17, version 32, address 0xfec30000, GSI 88-95
(XEN) [ 0.000000] ACPI: IOAPIC (id[0x12] address[0xfec38000] gsi_base[96])
(XEN) [ 0.000000] IOAPIC[8]: apic_id 18, version 32, address 0xfec38000, GSI 96-103
(XEN) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) [ 0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) [ 0.000000] ACPI: IRQ0 used by override.
(XEN) [ 0.000000] ACPI: IRQ2 used by override.
(XEN) [ 0.000000] ACPI: IRQ9 used by override.
(XEN) [ 0.000000] Enabling APIC mode: Phys. Using 9 I/O APICs
(XEN) [ 0.000000] ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [ 0.000000] [VT-D]Host address width 46
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = d37fc000
(XEN) [ 0.000000] [VT-D]drhd->address = d37fc000 iommu->reg = ffff82c000224000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:80:05.4
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.0
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.1
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.2
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.3
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.4
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.5
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.6
(XEN) [ 0.000000] [VT-D] endpoint: 0000:80:04.7
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = e0ffc000
(XEN) [ 0.000000] [VT-D]drhd->address = e0ffc000 iommu->reg = ffff82c000226000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:85:05.4
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = ee7fc000
(XEN) [ 0.000000] [VT-D]drhd->address = ee7fc000 iommu->reg = ffff82c000228000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:ae:05.4
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = fbffc000
(XEN) [ 0.000000] [VT-D]drhd->address = fbffc000 iommu->reg = ffff82c00022a000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:d7:05.4
(XEN) [ 0.000000] [VT-D] bridge: 0000:d7:00.0 start=d7 sec=d8 sub=d8
(XEN) [ 0.000000] [VT-D] bridge: 0000:d7:01.0 start=d7 sec=d9 sub=d9
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = aaffc000
(XEN) [ 0.000000] [VT-D]drhd->address = aaffc000 iommu->reg = ffff82c00022c000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:17:05.4
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = b87fc000
(XEN) [ 0.000000] [VT-D]drhd->address = b87fc000 iommu->reg = ffff82c00022e000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:3a:05.4
(XEN) [ 0.000000] [VT-D] bridge: 0000:3a:00.0 start=3a sec=3b sub=3e
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = c5ffc000
(XEN) [ 0.000000] [VT-D]drhd->address = c5ffc000 iommu->reg = ffff82c000230000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:5d:05.4
(XEN) [ 0.000000] [VT-D] bridge: 0000:5d:02.0 start=5d sec=5e sub=5e
(XEN) [ 0.000000] [VT-D] bridge: 0000:5d:03.0 start=5d sec=5f sub=5f
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_DRHD:
(XEN) [ 0.000000] [VT-D] dmaru->address = 9d7fc000
(XEN) [ 0.000000] [VT-D]drhd->address = 9d7fc000 iommu->reg = ffff82c000232000
(XEN) [ 0.000000] [VT-D]cap = 8d2078c106f0466 ecap = f020df
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:f0:1f.0
(XEN) [ 0.000000] [VT-D] IOAPIC: 0000:00:05.4
(XEN) [ 0.000000] [VT-D] MSI HPET: 0000:00:1f.0
(XEN) [ 0.000000] [VT-D] flags: INCLUDE_ALL
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_RMRR:
(XEN) [ 0.000000] [VT-D] RMRR address range 6be85000..6be87fff not in reserved memory; need "iommu_inclusive_mapping=1"?
(XEN) [ 0.000000] [VT-D] endpoint: 0000:00:14.0
(XEN) [ 0.000000] [VT-D]dmar.c:638: RMRR region: base_addr 6be85000 end_addr 6be87fff
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_ATSR:
(XEN) [ 0.000000] [VT-D] atsru->all_ports: 0
(XEN) [ 0.000000] [VT-D] bridge: 0000:3a:00.0 start=3a sec=3b sub=3e
(XEN) [ 0.000000] [VT-D] bridge: 0000:5d:02.0 start=5d sec=5e sub=5e
(XEN) [ 0.000000] [VT-D] bridge: 0000:5d:03.0 start=5d sec=5f sub=5f
(XEN) [ 0.000000] [VT-D]found ACPI_DMAR_ATSR:
(XEN) [ 0.000000] [VT-D] atsru->all_ports: 0
(XEN) [ 0.000000] [VT-D] bridge: 0000:d7:00.0 start=d7 sec=d8 sub=d8
(XEN) [ 0.000000] [VT-D] bridge: 0000:d7:01.0 start=d7 sec=d9 sub=d9
(XEN) [ 0.000000] Xen ERST support is initialized.
(XEN) [ 0.000000] HEST: Table parsing has been initialized
(XEN) [ 0.000000] ACPI: BGRT: invalidating v1 image at 0x67475000
(XEN) [ 0.000000] Using ACPI (MADT) for SMP configuration information
(XEN) [ 0.000000] SMP: Allowing 224 CPUs (112 hotplug CPUs)
(XEN) [ 0.000000] IRQ limits: 104 GSI, 21416 MSI/MSI-X
(XEN) [ 0.000000] Not enabling x2APIC (upon firmware request)
(XEN) [ 0.000000] xstate: size: 0xa88 and states: 0x2ff
(XEN) [ 0.000000] mce_intel.c:732: MCA capability: firstbank 0, 0 ext MSRs, BCAST, SER, CMCI
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU0 bank 6, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU0 bank 9, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU0 bank 10, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU0 bank 11, using 0x1
(XEN) [ 0.000000] CPU0: Intel machine check reporting enabled
(XEN) [ 0.000000] Using scheduler: SMP Credit Scheduler (credit)
(XEN) [ 0.000000] Platform timer is 23.999MHz HPET
(XEN) [ 10.392261] Detected 1795.796 MHz processor.
(XEN) [ 10.406138] Initing memory sharing.
(XEN) [ 10.411372] alt table ffff82d080431478 -> ffff82d080432ae0
(XEN) [ 10.418780] PCI: MCFG configuration 0: base 80000000 segment 0000 buses 00 - ff
(XEN) [ 10.428962] PCI: MCFG area at 80000000 reserved in E820
(XEN) [ 10.435818] PCI: Using MCFG for segment 0000 bus 00-ff
(XEN) [ 10.442898] Intel VT-d iommu 6 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.450795] Intel VT-d iommu 5 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.458699] Intel VT-d iommu 4 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.466607] Intel VT-d iommu 3 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.474581] Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.482488] Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.490391] Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.498291] Intel VT-d iommu 7 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.506195] Intel VT-d Snoop Control enabled.
(XEN) [ 10.512197] Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) [ 10.519152] Intel VT-d Queued Invalidation enabled.
(XEN) [ 10.525663] Intel VT-d Interrupt Remapping enabled.
(XEN) [ 10.532175] Intel VT-d Posted Interrupt not enabled.
(XEN) [ 10.538784] Intel VT-d Shared EPT tables enabled.
(XEN) [ 10.557822] I/O virtualisation enabled
(XEN) [ 10.563196] - Dom0 mode: Relaxed
(XEN) [ 10.568143] Interrupt remapping enabled
(XEN) [ 10.573600] nr_sockets: 5
(XEN) [ 10.577856] Enabled directed EOI with ioapic_ack_old on!
(XEN) [ 10.585787] ENABLING IO-APIC IRQs
(XEN) [ 10.590730] -> Using old ACK method
(XEN) [ 10.596437] ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) [ 10.806343] TSC deadline timer enabled
(XEN) [ 11.258347] Defaulting to alternative key handling; send 'A' to switch to normal mode.
(XEN) [ 11.268922] mwait-idle: MWAIT substates: 0x2020
(XEN) [ 11.270393] mwait-idle: v0.4.1 model 0x55
(XEN) [ 11.271857] mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) [ 11.273354] VMX: Supported advanced features:
(XEN) [ 11.274827] - APIC MMIO access virtualisation
(XEN) [ 11.276298] - APIC TPR shadow
(XEN) [ 11.278165] - Extended Page Tables (EPT)
(XEN) [ 11.279807] - Virtual-Processor Identifiers (VPID)
(XEN) [ 11.281285] - Virtual NMI
(XEN) [ 11.282733] - MSR direct-access bitmap
(XEN) [ 11.284203] - Unrestricted Guest
(XEN) [ 11.285663] - APIC Register Virtualization
(XEN) [ 11.287134] - Virtual Interrupt Delivery
(XEN) [ 11.288602] - Posted Interrupt Processing
(XEN) [ 11.290070] - VMCS shadowing
(XEN) [ 11.291519] - VM Functions
(XEN) [ 11.292968] - Virtualisation Exceptions
(XEN) [ 11.294434] - Page Modification Logging
(XEN) [ 11.295898] - TSC Scaling
(XEN) [ 11.297350] HVM: ASIDs enabled.
(XEN) [ 11.298810] HVM: VMX enabled
(XEN) [ 11.300259] HVM: Hardware Assisted Paging (HAP) detected
(XEN) [ 11.301738] HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 6, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 9, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 10, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 11, using 0x1
(XEN) [ 13.587194] Brought up 112 CPUs
(XEN) [ 14.109178] build-id: 3d8f735bdfb1af05757ffc57ad540b1cf0b7bd09
(XEN) [ 14.110658] Running stub recovery selftests...
(XEN) [ 14.112099] traps.c:3466: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080359cc2
(XEN) [ 14.114987] traps.c:813: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] -> ffff82d080359cc2
(XEN) [ 14.117867] traps.c:1215: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080359cc2
(XEN) [ 14.141414] TSC warp detected, disabling TSC_RELIABLE
(XEN) [ 14.142860] ACPI sleep modes: S3
(XEN) [ 14.144290] VPMU: disabled
(XEN) [ 14.145718] mcheck_poll: Machine check polling timer started.
(XEN) [ 14.147217] Dom0 has maximum 1448 PIRQs
(XEN) [ 14.151527] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:600
(XEN) [ 14.154404] ----[ Xen-4.9-unstable x86_64 debug=y Not tainted ]----
(XEN) [ 14.155867] CPU: 0
(XEN) [ 14.157286] RIP: e008:[<ffff82d0802ef735>] vioapic_init+0x110/0x167
(XEN) [ 14.158750] RFLAGS: 0000000000010287 CONTEXT: hypervisor
(XEN) [ 14.160203] rax: ffff830837c7fa00 rbx: 0000000000000009 rcx: 00000000c8381c70
(XEN) [ 14.163073] rdx: 0000000000000071 rsi: ffff830837c7e400 rdi: ffff83083fff7868
(XEN) [ 14.165937] rbp: ffff82d080457d28 rsp: ffff82d080457ce8 r8: ffff82e000000000
(XEN) [ 14.168797] r9: 0000000000000381 r10: ffff82d08045f400 r11: 0000000000000000
(XEN) [ 14.171657] r12: 0000000000000008 r13: ffff830837d29000 r14: 0000000000000058
(XEN) [ 14.174568] r15: ffff830837c7fb20 cr0: 000000008005003b cr4: 00000000003526e0
(XEN) [ 14.177437] cr3: 000000006f84c000 cr2: 0000000000000000
(XEN) [ 14.178887] ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) [ 14.181753] Xen code around <ffff82d0802ef735> (vioapic_init+0x110/0x167):
(XEN) [ 14.184609] 00 00 44 3b 70 40 74 02 <0f> 0b 8b 45 cc 41 89 85 b0 02 00 00 4c 89 ef e8
(XEN) [ 14.187473] Xen stack trace from rsp=ffff82d080457ce8:
(XEN) [ 14.188916] ffff82d08029e7de 0000000937c7f010 ffff82d080457d08 ffff830837d29000
(XEN) [ 14.191784] 0000000000000068 0000000000000001 0000000000000000 0000000000000000
(XEN) [ 14.194645] ffff82d080457d48 ffff82d0802de276 ffff830837d29000 0000000000000000
(XEN) [ 14.197507] ffff82d080457d78 ffff82d08026d593 ffff82d080457d78 ffff830837d29000
(XEN) [ 14.200371] 000000000000001f 0000000000000007 ffff82d080457de8 ffff82d080205226
(XEN) [ 14.203234] ffff82d0804380e0 0000000000000004 ffff82d080457eb4 0000000000000000
(XEN) [ 14.206097] ffff82d080457dc8 f7fa32231fcbfbff 000001212c100800 00000000000000e0
(XEN) [ 14.208956] ffff830838543850 00000000000000e0 ffff82d08043b780 000000000000006f
(XEN) [ 14.211817] ffff82d080457f08 ffff82d0803ee1be 000000000028fe80 000000000000015c
(XEN) [ 14.214739] 00000000000001df 0000000200000000 0000000000000002 0000000000000002
(XEN) [ 14.217598] 0000000000000002 0000000000000001 0000000000000001 0000000000000001
(XEN) [ 14.220459] 0000000000000001 0000000000000000 ffff82d080429a90 0000000000000017
(XEN) [ 14.223317] 0000001075ec7000 00000000013b7000 0000000001080000 0000000000000000
(XEN) [ 14.226178] ffff830000000002 ffff830000097c60 ffff830000097ee0 ffff830000097fb0
(XEN) [ 14.229039] 0000000000000000 0000001100000000 0000000800000000 000000010000006e
(XEN) [ 14.231901] 0000000000000003 00000000000002f8 0000000000000000 0000000000000000
(XEN) [ 14.234770] 00000000000000e8 0000000068924a50 000000006c5b8a28 000000006c5b8a40
(XEN) [ 14.237640] 0000000000000000 ffff82d0802000f3 0000000000000000 0000000000000000
(XEN) [ 14.240504] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [ 14.243374] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [ 14.246245] Xen call trace:
(XEN) [ 14.247668] [<ffff82d0802ef735>] vioapic_init+0x110/0x167
(XEN) [ 14.249126] [<ffff82d0802de276>] hvm_domain_initialise+0x268/0x365
(XEN) [ 14.250593] [<ffff82d08026d593>] arch_domain_create+0x4b0/0x6ac
(XEN) [ 14.252055] [<ffff82d080205226>] domain_create+0x3d8/0x593
(XEN) [ 14.253578] [<ffff82d0803ee1be>] __start_xen+0x20fa/0x24c3
(XEN) [ 14.255029] [<ffff82d0802000f3>] __high_start+0x53/0x58
(XEN) [ 14.256481]
(XEN) [ 14.510271]
(XEN) [ 14.513527] ****************************************
(XEN) [ 14.520279] Panic on CPU 0:
(XEN) [ 14.524726] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:600
(XEN) [ 14.535470] ****************************************
(XEN) [ 14.542091]
(XEN) [ 14.545230] Reboot in five seconds...
(XEN) [ 19.553630] Resetting with ACPI MEMORY or I/O RESET_REG.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 7:47 ` Roger Pau Monné
@ 2017-04-17 1:03 ` Chao Gao
2017-04-17 8:38 ` Roger Pau Monné
0 siblings, 1 reply; 16+ messages in thread
From: Chao Gao @ 2017-04-17 1:03 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote:
>On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
>> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
>> >Hello,
>> >
>> >Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
>> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
>> >completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
>> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
>> >
>> >I'm not able to debug that in any meaningful way because the box seems to lock
>> >up completely, even the watchdog NMI stops working. Here is the boot log, up to
>> >the point where it freezes:
>>
>> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
>>
>---8<---
>diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
>index 527ac2aadd..1df7710041 100644
>--- a/xen/arch/x86/hvm/vioapic.c
>+++ b/xen/arch/x86/hvm/vioapic.c
>@@ -625,6 +625,9 @@ int vioapic_init(struct domain *d)
> nr_gsis += nr_pins;
> }
>
>+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
>+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
>+
> ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
>
> d->arch.hvm_domain.nr_vioapics = nr_vioapics;
With the above patch,
(XEN) [ 10.420001] PCI: MCFG area at 80000000 reserved in E820
(XEN) [ 10.426854] PCI: Using MCFG for segment 0000 bus 00-ff
(XEN) [ 10.433952] Intel VT-d iommu 6 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.441856] Intel VT-d iommu 5 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.449759] Intel VT-d iommu 4 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.457671] Intel VT-d iommu 3 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.465585] Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.473485] Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.481394] Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.489299] Intel VT-d iommu 7 supported page sizes: 4kB, 2MB, 1GB.
(XEN) [ 10.497196] Intel VT-d Snoop Control enabled.
(XEN) [ 10.503196] Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) [ 10.510145] Intel VT-d Queued Invalidation enabled.
(XEN) [ 10.516646] Intel VT-d Interrupt Remapping enabled.
(XEN) [ 10.523173] Intel VT-d Posted Interrupt not enabled.
(XEN) [ 10.529775] Intel VT-d Shared EPT tables enabled.
(XEN) [ 10.548815] I/O virtualisation enabled
(XEN) [ 10.554186] - Dom0 mode: Relaxed
(XEN) [ 10.559264] Interrupt remapping enabled
(XEN) [ 10.564854] nr_sockets: 5
(XEN) [ 10.569231] Enabled directed EOI with ioapic_ack_old on!
(XEN) [ 10.577294] ENABLING IO-APIC IRQs
(XEN) [ 10.582245] -> Using old ACK method
(XEN) [ 10.587967] ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) [ 10.797645] TSC deadline timer enabled
(XEN) [ 10.887286] Defaulting to alternative key handling; send 'A' to switch to normal mode.
(XEN) [ 10.897864] mwait-idle: MWAIT substates: 0x2020
(XEN) [ 10.899335] mwait-idle: v0.4.1 model 0x55
(XEN) [ 10.900799] mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) [ 10.902304] VMX: Supported advanced features:
(XEN) [ 10.903781] - APIC MMIO access virtualisation
(XEN) [ 10.905258] - APIC TPR shadow
(XEN) [ 10.907138] - Extended Page Tables (EPT)
(XEN) [ 10.908782] - Virtual-Processor Identifiers (VPID)
(XEN) [ 10.910262] - Virtual NMI
(XEN) [ 10.911719] - MSR direct-access bitmap
(XEN) [ 10.913188] - Unrestricted Guest
(XEN) [ 10.914650] - APIC Register Virtualization
(XEN) [ 10.916126] - Virtual Interrupt Delivery
(XEN) [ 10.917596] - Posted Interrupt Processing
(XEN) [ 10.919066] - VMCS shadowing
(XEN) [ 10.920519] - VM Functions
(XEN) [ 10.921976] - Virtualisation Exceptions
(XEN) [ 10.923448] - Page Modification Logging
(XEN) [ 10.924918] - TSC Scaling
(XEN) [ 10.926371] HVM: ASIDs enabled.
(XEN) [ 10.927829] HVM: VMX enabled
(XEN) [ 10.929278] HVM: Hardware Assisted Paging (HAP) detected
(XEN) [ 10.930762] HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 6, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 9, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 10, using 0x1
(XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 11, using 0x1
(XEN) [ 13.216648] Brought up 112 CPUs
(XEN) [ 13.739330] build-id: dc4540250abe5d96614d340c67069e390c37c21c
(XEN) [ 13.740816] Running stub recovery selftests...
(XEN) [ 13.742258] traps.c:3466: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080359cf2
(XEN) [ 13.745155] traps.c:813: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] -> ffff82d080359cf2
(XEN) [ 13.748046] traps.c:1215: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080359cf2
(XEN) [ 13.771452] TSC warp detected, disabling TSC_RELIABLE
(XEN) [ 13.772897] ACPI sleep modes: S3
(XEN) [ 13.774329] VPMU: disabled
(XEN) [ 13.775757] mcheck_poll: Machine check polling timer started.
(XEN) [ 13.777246] Dom0 has maximum 1448 PIRQs
(XEN) [ 13.779426] domain nr_gsis: 104 vioapic gsis: 88 nr_irqs_gsi: 104 highest_gsi: 103
(XEN) [ 13.782371] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:603
(XEN) [ 13.785249] ----[ Xen-4.9-unstable x86_64 debug=y Not tainted ]----
(XEN) [ 13.786715] CPU: 0
(XEN) [ 13.788137] RIP: e008:[<ffff82d0802ef764>] vioapic_init+0x13f/0x196
(XEN) [ 13.789605] RFLAGS: 0000000000010287 CONTEXT: hypervisor
(XEN) [ 13.791065] rax: ffff830837c7ea00 rbx: 0000000000000009 rcx: 0000000000000000
(XEN) [ 13.793948] rdx: ffff82d080457fff rsi: 000000000000000a rdi: ffff82d08044d6b8
(XEN) [ 13.796819] rbp: ffff82d080457d28 rsp: ffff82d080457ce8 r8: ffff83083e000000
(XEN) [ 13.799690] r9: 0000000000000006 r10: 00000000000be4a2 r11: 0000000000000006
(XEN) [ 13.802565] r12: 0000000000000008 r13: ffff830837d2e000 r14: 0000000000000058
(XEN) [ 13.805440] r15: ffff830837c7eb20 cr0: 000000008005003b cr4: 00000000003526e0
(XEN) [ 13.808316] cr3: 000000006f84c000 cr2: 0000000000000000
(XEN) [ 13.809768] ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) [ 13.812648] Xen code around <ffff82d0802ef764> (vioapic_init+0x13f/0x196):
(XEN) [ 13.815516] 00 00 44 3b 70 40 74 02 <0f> 0b 8b 45 cc 41 89 85 b0 02 00 00 4c 89 ef e8
(XEN) [ 13.818458] Xen stack trace from rsp=ffff82d080457ce8:
(XEN) [ 13.819902] ffff82d08029e7de 0000000937c7e010 ffff82d080457d08 ffff830837d2e000
(XEN) [ 13.822785] 0000000000000068 0000000000000001 0000000000000000 0000000000000000
(XEN) [ 13.825660] ffff82d080457d48 ffff82d0802de276 ffff830837d2e000 0000000000000000
(XEN) [ 13.828535] ffff82d080457d78 ffff82d08026d593 ffff82d080457d78 ffff830837d2e000
(XEN) [ 13.831410] 000000000000001f 0000000000000007 ffff82d080457de8 ffff82d080205226
(XEN) [ 13.834286] ffff82d0804380e0 0000000000000004 ffff82d080457eb4 0000000000000000
(XEN) [ 13.837164] ffff82d080457dc8 f7fa32231fcbfbff 000001212c100800 00000000000000e0
(XEN) [ 13.840038] ffff830838543850 00000000000000e0 ffff82d08043b780 000000000000006f
(XEN) [ 13.842917] ffff82d080457f08 ffff82d0803ee1be 000000000028fe80 000000000000015c
(XEN) [ 13.845795] 00000000000001df 0000000200000000 0000000000000002 0000000000000002
(XEN) [ 13.848669] 0000000000000002 0000000000000001 0000000000000001 0000000000000001
(XEN) [ 13.851544] 0000000000000001 0000000000000000 ffff82d080429a90 0000000000000017
(XEN) [ 13.854486] 0000001075ec7000 00000000013b7000 0000000001080000 0000000000000000
(XEN) [ 13.857356] ffff830000000002 ffff830000097c60 ffff830000097ee0 ffff830000097fb0
(XEN) [ 13.860226] 0000000000000000 0000001100000000 0000000800000000 000000010000006e
(XEN) [ 13.863101] 0000000000000003 00000000000002f8 0000000000000000 0000000000000000
(XEN) [ 13.865975] 00000000000000e8 0000000068924a50 000000006c5b8a28 000000006c5b8a40
(XEN) [ 13.868847] 0000000000000000 ffff82d0802000f3 0000000000000000 0000000000000000
(XEN) [ 13.871723] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [ 13.874600] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [ 13.877472] Xen call trace:
(XEN) [ 13.878892] [<ffff82d0802ef764>] vioapic_init+0x13f/0x196
(XEN) [ 13.880353] [<ffff82d0802de276>] hvm_domain_initialise+0x268/0x365
(XEN) [ 13.881825] [<ffff82d08026d593>] arch_domain_create+0x4b0/0x6ac
(XEN) [ 13.883294] [<ffff82d080205226>] domain_create+0x3d8/0x593
(XEN) [ 13.884759] [<ffff82d0803ee1be>] __start_xen+0x20fa/0x24c3
(XEN) [ 13.886221] [<ffff82d0802000f3>] __high_start+0x53/0x58
(XEN) [ 13.887681]
(XEN) [ 14.148367]
(XEN) [ 14.151511] ****************************************
(XEN) [ 14.158124] Panic on CPU 0:
(XEN) [ 14.162565] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:603
(XEN) [ 14.173326] ****************************************
(XEN) [ 14.179947]
(XEN) [ 14.183083] Reboot in five seconds...
(XEN) [ 19.190218] Resetting with ACPI MEMORY or I/O RESET_REG.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 8:38 ` Roger Pau Monné
@ 2017-04-17 2:49 ` Chao Gao
2017-04-17 10:38 ` Roger Pau Monné
0 siblings, 1 reply; 16+ messages in thread
From: Chao Gao @ 2017-04-17 2:49 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 09:38:54AM +0100, Roger Pau Monné wrote:
>On Mon, Apr 17, 2017 at 09:03:12AM +0800, Chao Gao wrote:
>> On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote:
>> >On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
>> >> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
>> >> >Hello,
>> >> >
>> >> >Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
>> >> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
>> >> >completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
>> >> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
>> >> >
>> >> >I'm not able to debug that in any meaningful way because the box seems to lock
>> >> >up completely, even the watchdog NMI stops working. Here is the boot log, up to
>> >> >the point where it freezes:
>> >>
>> >> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
>> >>
>
>It seems like we are not properly adding/accounting the vIO APICs, but I cannot
>really see how. I have another patch for you to try below.
>
>Thanks, Roger.
>
>---8<---
> diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
>index 527ac2aadd..40075e2756 100644
>--- a/xen/arch/x86/hvm/vioapic.c
>+++ b/xen/arch/x86/hvm/vioapic.c
>@@ -610,11 +610,15 @@ int vioapic_init(struct domain *d)
> xzalloc_array(struct hvm_vioapic *, nr_vioapics)) == NULL) )
> return -ENOMEM;
>
>+ printk("Adding %u vIO APICs\n", nr_vioapics);
>+
> for ( i = 0; i < nr_vioapics; i++ )
> {
> unsigned int nr_pins = is_hardware_domain(d) ? nr_ioapic_entries[i] :
> ARRAY_SIZE(domain_vioapic(d, 0)->domU.redirtbl);
>
>+ printk("vIO APIC %u has %u pins\n", i, nr_pins);
>+
> if ( (domain_vioapic(d, i) =
> xmalloc_bytes(hvm_vioapic_size(nr_pins))) == NULL )
> {
>@@ -623,8 +627,12 @@ int vioapic_init(struct domain *d)
> }
> domain_vioapic(d, i)->nr_pins = nr_pins;
> nr_gsis += nr_pins;
>+ printk("nr_gsis: %u\n", nr_gsis);
> }
>
>+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
>+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
>+
> ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
>
> d->arch.hvm_domain.nr_vioapics = nr_vioapics;
>
Please Cc or To me. Is there some holes in all physical IOAPICs gsi ranges?
with the above patch,
(XEN) [ 14.262237] Dom0 has maximum 1448 PIRQs
(XEN) [ 14.264413] Adding 9 vIO APICs
(XEN) [ 14.265827] vIO APIC 0 has 24 pins
(XEN) [ 14.267256] nr_gsis: 24
(XEN) [ 14.268673] vIO APIC 1 has 8 pins
(XEN) [ 14.270175] nr_gsis: 32
(XEN) [ 14.271589] vIO APIC 2 has 8 pins
(XEN) [ 14.273011] nr_gsis: 40
(XEN) [ 14.274434] vIO APIC 3 has 8 pins
(XEN) [ 14.275864] nr_gsis: 48
(XEN) [ 14.277283] vIO APIC 4 has 8 pins
(XEN) [ 14.278709] nr_gsis: 56
(XEN) [ 14.280127] vIO APIC 5 has 8 pins
(XEN) [ 14.281561] nr_gsis: 64
(XEN) [ 14.282986] vIO APIC 6 has 8 pins
(XEN) [ 14.284417] nr_gsis: 72
(XEN) [ 14.285837] vIO APIC 7 has 8 pins
(XEN) [ 14.287262] nr_gsis: 80
(XEN) [ 14.288683] vIO APIC 8 has 8 pins
(XEN) [ 14.290114] nr_gsis: 88
(XEN) [ 14.291538] domain nr_gsis: 104 vioapic gsis: 88 nr_irqs_gsi: 104 highest_gsi: 103
(XEN) [ 14.294417] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:608
(XEN) [ 14.297282] ----[ Xen-4.9-unstable x86_64 debug=y Not tainted ]----
(XEN) [ 14.298743] CPU: 0
(XEN) [ 14.300161] RIP: e008:[<ffff82d0802ef7ab>] vioapic_init+0x186/0x1dd
(XEN) [ 14.301633] RFLAGS: 0000000000010287 CONTEXT: hypervisor
(XEN) [ 14.303094] rax: ffff830837c7ea00 rbx: 0000000000000009 rcx: 0000000000000000
(XEN) [ 14.305976] rdx: ffff82d080457fff rsi: 000000000000000a rdi: ffff82d08044d6b8
(XEN) [ 14.308851] rbp: ffff82d080457d28 rsp: ffff82d080457ce8 r8: ffff83083e000000
(XEN) [ 14.311781] r9: 0000000000000006 r10: 00000000000472d2 r11: 0000000000000006
(XEN) [ 14.314654] r12: 0000000000000008 r13: ffff830837d2e000 r14: 0000000000000058
(XEN) [ 14.317528] r15: ffff830837c7eb20 cr0: 000000008005003b cr4: 00000000003526e0
(XEN) [ 14.320403] cr3: 000000006f84c000 cr2: 0000000000000000
(XEN) [ 14.321855] ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) [ 14.324734] Xen code around <ffff82d0802ef7ab> (vioapic_init+0x186/0x1dd):
(XEN) [ 14.327591] 00 00 44 3b 70 40 74 02 <0f> 0b 8b 45 cc 41 89 85 b0 02 00 00 4c 89 ef e8
(XEN) [ 14.330458] Xen stack trace from rsp=ffff82d080457ce8:
(XEN) [ 14.331908] ffff82d08029e7de 0000000937c7e010 ffff82d080457d08 ffff830837d2e000
(XEN) [ 14.334790] 0000000000000068 0000000000000001 0000000000000000 0000000000000000
(XEN) [ 14.337661] ffff82d080457d48 ffff82d0802de276 ffff830837d2e000 0000000000000000
(XEN) [ 14.340526] ffff82d080457d78 ffff82d08026d593 ffff82d080457d78 ffff830837d2e000
(XEN) [ 14.343402] 000000000000001f 0000000000000007 ffff82d080457de8 ffff82d080205226
(XEN) [ 14.346270] ffff82d0804380e0 0000000000000004 ffff82d080457eb4 0000000000000000
(XEN) [ 14.349202] ffff82d080457dc8 f7fa32231fcbfbff 000001212c100800 00000000000000e0
(XEN) [ 14.352071] ffff830838543850 00000000000000e0 ffff82d08043b780 000000000000006f
(XEN) [ 14.354944] ffff82d080457f08 ffff82d0803ee1be 000000000028fe80 000000000000015c
(XEN) [ 14.357815] 00000000000001df 0000000200000000 0000000000000002 0000000000000002
(XEN) [ 14.360683] 0000000000000002 0000000000000001 0000000000000001 0000000000000001
(XEN) [ 14.363674] 0000000000000001 0000000000000000 ffff82d080429a90 0000000000000017
(XEN) [ 14.366548] 0000001075ec7000 00000000013b7000 0000000001080000 0000000000000000
(XEN) [ 14.369412] ffff830000000002 ffff830000097c60 ffff830000097ee0 ffff830000097fb0
(XEN) [ 14.372287] 0000000000000000 0000001100000000 0000000800000000 000000010000006e
(XEN) [ 14.375164] 0000000000000003 00000000000002f8 0000000000000000 0000000000000000
(XEN) [ 14.378035] 00000000000000e8 0000000068924a50 000000006c5b8a28 000000006c5b8a40
(XEN) [ 14.380900] 0000000000000000 ffff82d0802000f3 0000000000000000 0000000000000000
(XEN) [ 14.383764] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [ 14.386673] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) [ 14.389535] Xen call trace:
(XEN) [ 14.390952] [<ffff82d0802ef7ab>] vioapic_init+0x186/0x1dd
(XEN) [ 14.392408] [<ffff82d0802de276>] hvm_domain_initialise+0x268/0x365
(XEN) [ 14.393875] [<ffff82d08026d593>] arch_domain_create+0x4b0/0x6ac
(XEN) [ 14.395339] [<ffff82d080205226>] domain_create+0x3d8/0x593
(XEN) [ 14.396798] [<ffff82d0803ee1be>] __start_xen+0x20fa/0x24c3
(XEN) [ 14.398255] [<ffff82d0802000f3>] __high_start+0x53/0x58
(XEN) [ 14.399708]
(XEN) [ 14.697858]
(XEN) [ 14.700994] ****************************************
(XEN) [ 14.707623] Panic on CPU 0:
(XEN) [ 14.712067] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:608
(XEN) [ 14.722814] ****************************************
(XEN) [ 14.729433]
(XEN) [ 14.732571] Reboot in five seconds...
(XEN) [ 19.739647] Resetting with ACPI MEMORY or I/O RESET_REG.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 10:38 ` Roger Pau Monné
@ 2017-04-17 5:12 ` Chao Gao
2017-04-17 12:21 ` Roger Pau Monné
0 siblings, 1 reply; 16+ messages in thread
From: Chao Gao @ 2017-04-17 5:12 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 11:38:33AM +0100, Roger Pau Monné wrote:
>On Mon, Apr 17, 2017 at 10:49:45AM +0800, Chao Gao wrote:
>> On Mon, Apr 17, 2017 at 09:38:54AM +0100, Roger Pau Monné wrote:
>> >On Mon, Apr 17, 2017 at 09:03:12AM +0800, Chao Gao wrote:
>> >> On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote:
>> >> >On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
>> >> >> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
>> >> >> >Hello,
>> >> >> >
>> >> >> >Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
>> >> >> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
>> >> >> >completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
>> >> >> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
>> >> >> >
>> >> >> >I'm not able to debug that in any meaningful way because the box seems to lock
>> >> >> >up completely, even the watchdog NMI stops working. Here is the boot log, up to
>> >> >> >the point where it freezes:
>> >> >>
>> >> >> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
>> >> >>
>> >
>> >It seems like we are not properly adding/accounting the vIO APICs, but I cannot
>> >really see how. I have another patch for you to try below.
>> >
>> >Thanks, Roger.
>> >
>> >---8<---
>> > diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
>> >index 527ac2aadd..40075e2756 100644
>> >--- a/xen/arch/x86/hvm/vioapic.c
>> >+++ b/xen/arch/x86/hvm/vioapic.c
>> >@@ -610,11 +610,15 @@ int vioapic_init(struct domain *d)
>> > xzalloc_array(struct hvm_vioapic *, nr_vioapics)) == NULL) )
>> > return -ENOMEM;
>> >
>> >+ printk("Adding %u vIO APICs\n", nr_vioapics);
>> >+
>> > for ( i = 0; i < nr_vioapics; i++ )
>> > {
>> > unsigned int nr_pins = is_hardware_domain(d) ? nr_ioapic_entries[i] :
>> > ARRAY_SIZE(domain_vioapic(d, 0)->domU.redirtbl);
>> >
>> >+ printk("vIO APIC %u has %u pins\n", i, nr_pins);
>> >+
>> > if ( (domain_vioapic(d, i) =
>> > xmalloc_bytes(hvm_vioapic_size(nr_pins))) == NULL )
>> > {
>> >@@ -623,8 +627,12 @@ int vioapic_init(struct domain *d)
>> > }
>> > domain_vioapic(d, i)->nr_pins = nr_pins;
>> > nr_gsis += nr_pins;
>> >+ printk("nr_gsis: %u\n", nr_gsis);
>> > }
>> >
>> >+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
>> >+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
>> >+
>> > ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
>> >
>> > d->arch.hvm_domain.nr_vioapics = nr_vioapics;
>> >
>>
>> Please Cc or To me. Is there some holes in all physical IOAPICs gsi ranges?
>
>That's weird, my MUA (Mutt) seems to automatically remove your address from the
>"To:" field. I have no idea why it does that.
>
>So yes, your box has as GSI gap which is not handled by any IO APIC. TBH, I
>didn't even knew that was possible. In any case, patch below should solve it.
>
>---8<---
>commit f52d05fca03440d771eb56077c9d60bb630eb423
>diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
>index 5157db7a4e..ec87a97651 100644
>--- a/xen/arch/x86/hvm/vioapic.c
>+++ b/xen/arch/x86/hvm/vioapic.c
>@@ -64,37 +64,23 @@ static struct hvm_vioapic *addr_vioapic(const struct domain *d,
> struct hvm_vioapic *gsi_vioapic(const struct domain *d, unsigned int gsi,
> unsigned int *pin)
> {
>- unsigned int i, base_gsi = 0;
>+ unsigned int i;
>
> for ( i = 0; i < d->arch.hvm_domain.nr_vioapics; i++ )
> {
> struct hvm_vioapic *vioapic = domain_vioapic(d, i);
>
>- if ( gsi >= base_gsi && gsi < base_gsi + vioapic->nr_pins )
>+ if ( gsi >= vioapic->base_gsi &&
>+ gsi < vioapic->base_gsi + vioapic->nr_pins )
> {
>- *pin = gsi - base_gsi;
>+ *pin = gsi - vioapic->base_gsi;
> return vioapic;
> }
>-
>- base_gsi += vioapic->nr_pins;
> }
>
> return NULL;
> }
>
>-static unsigned int base_gsi(const struct domain *d,
>- const struct hvm_vioapic *vioapic)
>-{
>- unsigned int nr_vioapics = d->arch.hvm_domain.nr_vioapics;
>- unsigned int base_gsi = 0, i = 0;
>- const struct hvm_vioapic *tmp;
>-
>- while ( i < nr_vioapics && (tmp = domain_vioapic(d, i++)) != vioapic )
>- base_gsi += tmp->nr_pins;
>-
>- return base_gsi;
>-}
>-
> static uint32_t vioapic_read_indirect(const struct hvm_vioapic *vioapic)
> {
> uint32_t result = 0;
>@@ -180,7 +166,7 @@ static void vioapic_write_redirent(
> struct hvm_irq *hvm_irq = hvm_domain_irq(d);
> union vioapic_redir_entry *pent, ent;
> int unmasked = 0;
>- unsigned int gsi = base_gsi(d, vioapic) + idx;
>+ unsigned int gsi = vioapic->base_gsi + idx;
>
> spin_lock(&d->arch.hvm_domain.irq_lock);
>
>@@ -340,7 +326,7 @@ static void vioapic_deliver(struct hvm_vioapic *vioapic, unsigned int pin)
> struct domain *d = vioapic_domain(vioapic);
> struct vlapic *target;
> struct vcpu *v;
>- unsigned int irq = base_gsi(d, vioapic) + pin;
>+ unsigned int irq = vioapic->base_gsi + pin;
>
> ASSERT(spin_is_locked(&d->arch.hvm_domain.irq_lock));
>
>@@ -451,7 +437,7 @@ void vioapic_update_EOI(struct domain *d, u8 vector)
> {
> struct hvm_irq *hvm_irq = hvm_domain_irq(d);
> union vioapic_redir_entry *ent;
>- unsigned int i, base_gsi = 0;
>+ unsigned int i;
>
> ASSERT(has_vioapic(d));
>
>@@ -473,19 +459,18 @@ void vioapic_update_EOI(struct domain *d, u8 vector)
> if ( iommu_enabled )
> {
> spin_unlock(&d->arch.hvm_domain.irq_lock);
>- hvm_dpci_eoi(d, base_gsi + pin, ent);
>+ hvm_dpci_eoi(d, vioapic->base_gsi + pin, ent);
> spin_lock(&d->arch.hvm_domain.irq_lock);
> }
>
> if ( (ent->fields.trig_mode == VIOAPIC_LEVEL_TRIG) &&
> !ent->fields.mask &&
>- hvm_irq->gsi_assert_count[base_gsi + pin] )
>+ hvm_irq->gsi_assert_count[vioapic->base_gsi + pin] )
> {
> ent->fields.remote_irr = 1;
> vioapic_deliver(vioapic, pin);
> }
> }
>- base_gsi += vioapic->nr_pins;
> }
>
> spin_unlock(&d->arch.hvm_domain.irq_lock);
>@@ -554,6 +539,7 @@ void vioapic_reset(struct domain *d)
> {
> vioapic->base_address = mp_ioapics[i].mpc_apicaddr;
> vioapic->id = mp_ioapics[i].mpc_apicid;
>+ vioapic->base_gsi = io_apic_gsi_base(i);
> }
> vioapic->nr_pins = nr_pins;
> vioapic->domain = d;
>@@ -601,7 +587,12 @@ int vioapic_init(struct domain *d)
> nr_gsis += nr_pins;
> }
>
>- ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
>+ /*
>+ * NB: hvm_domain_irq(d)->nr_gsis is actually the highest GSI + 1, but
>+ * there might be holes in this range (ie: GSIs that don't belong to any
>+ * vIO APIC).
>+ */
>+ ASSERT(hvm_domain_irq(d)->nr_gsis >= nr_gsis);
>
> d->arch.hvm_domain.nr_vioapics = nr_vioapics;
> vioapic_reset(d);
>diff --git a/xen/include/asm-x86/hvm/vioapic.h b/xen/include/asm-x86/hvm/vioapic.h
>index 8ec91d2bd1..2ceb60eaef 100644
>--- a/xen/include/asm-x86/hvm/vioapic.h
>+++ b/xen/include/asm-x86/hvm/vioapic.h
>@@ -50,6 +50,7 @@
> struct hvm_vioapic {
> struct domain *domain;
> uint32_t nr_pins;
>+ unsigned int base_gsi;
> union {
> XEN_HVM_VIOAPIC(,);
> struct hvm_hw_vioapic domU;
>
It works. I can test for you when you send out a formal patch.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 12:21 ` Roger Pau Monné
@ 2017-04-17 5:47 ` Chao Gao
2017-04-17 12:57 ` Roger Pau Monné
0 siblings, 1 reply; 16+ messages in thread
From: Chao Gao @ 2017-04-17 5:47 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 01:21:01PM +0100, Roger Pau Monné wrote:
>On Mon, Apr 17, 2017 at 01:12:27PM +0800, Chao Gao wrote:
>[...]
>> It works. I can test for you when you send out a formal patch.
>
>Thanks for the testing, will send formal patch shortly.
>
>Have you been able to reproduce the IOMMU issue with that, or you just hit the
>panic at the end of PVH Dom0 build?
No, I haven't. The output is some like ELFxxx not found, I think, due to lack
of pvh domain0 kernel. As mentioned before, my platform is skylake.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 12:57 ` Roger Pau Monné
@ 2017-04-17 6:18 ` Chao Gao
0 siblings, 0 replies; 16+ messages in thread
From: Chao Gao @ 2017-04-17 6:18 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 01:57:10PM +0100, Roger Pau Monné wrote:
>On Mon, Apr 17, 2017 at 01:47:47PM +0800, Chao Gao wrote:
>> On Mon, Apr 17, 2017 at 01:21:01PM +0100, Roger Pau Monné wrote:
>> >On Mon, Apr 17, 2017 at 01:12:27PM +0800, Chao Gao wrote:
>> >[...]
>> >> It works. I can test for you when you send out a formal patch.
>> >
>> >Thanks for the testing, will send formal patch shortly.
>> >
>> >Have you been able to reproduce the IOMMU issue with that, or you just hit the
>> >panic at the end of PVH Dom0 build?
>>
>> No, I haven't. The output is some like ELFxxx not found, I think, due to lack
>> of pvh domain0 kernel. As mentioned before, my platform is skylake.
>
>Right, if you get to the ELF stuff it means the IOMMU has been initialized
>successfully. Skylake is post-haswell, so I don't think it's going to exhibit
>those issues. Is there any chance you can test on something older
>(pre-haswell?).
I am not sure that I can find a pre-haswell machine. Will try later.
Thanks
Chao
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-16 23:32 ` Chao Gao
@ 2017-04-17 7:47 ` Roger Pau Monné
2017-04-17 1:03 ` Chao Gao
0 siblings, 1 reply; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-17 7:47 UTC (permalink / raw)
To: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
> >Hello,
> >
> >Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
> >completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
> >
> >I'm not able to debug that in any meaningful way because the box seems to lock
> >up completely, even the watchdog NMI stops working. Here is the boot log, up to
> >the point where it freezes:
>
> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
>
[...]
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x08] address[0xfec00000] gsi_base[0])
> (XEN) [ 0.000000] IOAPIC[0]: apic_id 8, version 32, address 0xfec00000, GSI 0-23
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x09] address[0xfec01000] gsi_base[24])
> (XEN) [ 0.000000] IOAPIC[1]: apic_id 9, version 32, address 0xfec01000, GSI 24-31
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x0a] address[0xfec08000] gsi_base[32])
> (XEN) [ 0.000000] IOAPIC[2]: apic_id 10, version 32, address 0xfec08000, GSI 32-39
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x0b] address[0xfec10000] gsi_base[40])
> (XEN) [ 0.000000] IOAPIC[3]: apic_id 11, version 32, address 0xfec10000, GSI 40-47
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x0c] address[0xfec18000] gsi_base[48])
> (XEN) [ 0.000000] IOAPIC[4]: apic_id 12, version 32, address 0xfec18000, GSI 48-55
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x0f] address[0xfec20000] gsi_base[72])
> (XEN) [ 0.000000] IOAPIC[5]: apic_id 15, version 32, address 0xfec20000, GSI 72-79
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x10] address[0xfec28000] gsi_base[80])
> (XEN) [ 0.000000] IOAPIC[6]: apic_id 16, version 32, address 0xfec28000, GSI 80-87
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x11] address[0xfec30000] gsi_base[88])
> (XEN) [ 0.000000] IOAPIC[7]: apic_id 17, version 32, address 0xfec30000, GSI 88-95
> (XEN) [ 0.000000] ACPI: IOAPIC (id[0x12] address[0xfec38000] gsi_base[96])
> (XEN) [ 0.000000] IOAPIC[8]: apic_id 18, version 32, address 0xfec38000, GSI 96-103
[...]
> (XEN) [ 0.000000] IRQ limits: 104 GSI, 21416 MSI/MSI-X
[...]
> (XEN) [ 14.147217] Dom0 has maximum 1448 PIRQs
> (XEN) [ 14.151527] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:600
> (XEN) [ 14.154404] ----[ Xen-4.9-unstable x86_64 debug=y Not tainted ]----
> (XEN) [ 14.155867] CPU: 0
> (XEN) [ 14.157286] RIP: e008:[<ffff82d0802ef735>] vioapic_init+0x110/0x167
> (XEN) [ 14.158750] RFLAGS: 0000000000010287 CONTEXT: hypervisor
> (XEN) [ 14.160203] rax: ffff830837c7fa00 rbx: 0000000000000009 rcx: 00000000c8381c70
> (XEN) [ 14.163073] rdx: 0000000000000071 rsi: ffff830837c7e400 rdi: ffff83083fff7868
> (XEN) [ 14.165937] rbp: ffff82d080457d28 rsp: ffff82d080457ce8 r8: ffff82e000000000
> (XEN) [ 14.168797] r9: 0000000000000381 r10: ffff82d08045f400 r11: 0000000000000000
> (XEN) [ 14.171657] r12: 0000000000000008 r13: ffff830837d29000 r14: 0000000000000058
> (XEN) [ 14.174568] r15: ffff830837c7fb20 cr0: 000000008005003b cr4: 00000000003526e0
> (XEN) [ 14.177437] cr3: 000000006f84c000 cr2: 0000000000000000
> (XEN) [ 14.178887] ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) [ 14.181753] Xen code around <ffff82d0802ef735> (vioapic_init+0x110/0x167):
> (XEN) [ 14.184609] 00 00 44 3b 70 40 74 02 <0f> 0b 8b 45 cc 41 89 85 b0 02 00 00 4c 89 ef e8
> (XEN) [ 14.187473] Xen stack trace from rsp=ffff82d080457ce8:
> (XEN) [ 14.188916] ffff82d08029e7de 0000000937c7f010 ffff82d080457d08 ffff830837d29000
> (XEN) [ 14.191784] 0000000000000068 0000000000000001 0000000000000000 0000000000000000
> (XEN) [ 14.194645] ffff82d080457d48 ffff82d0802de276 ffff830837d29000 0000000000000000
> (XEN) [ 14.197507] ffff82d080457d78 ffff82d08026d593 ffff82d080457d78 ffff830837d29000
> (XEN) [ 14.200371] 000000000000001f 0000000000000007 ffff82d080457de8 ffff82d080205226
> (XEN) [ 14.203234] ffff82d0804380e0 0000000000000004 ffff82d080457eb4 0000000000000000
> (XEN) [ 14.206097] ffff82d080457dc8 f7fa32231fcbfbff 000001212c100800 00000000000000e0
> (XEN) [ 14.208956] ffff830838543850 00000000000000e0 ffff82d08043b780 000000000000006f
> (XEN) [ 14.211817] ffff82d080457f08 ffff82d0803ee1be 000000000028fe80 000000000000015c
> (XEN) [ 14.214739] 00000000000001df 0000000200000000 0000000000000002 0000000000000002
> (XEN) [ 14.217598] 0000000000000002 0000000000000001 0000000000000001 0000000000000001
> (XEN) [ 14.220459] 0000000000000001 0000000000000000 ffff82d080429a90 0000000000000017
> (XEN) [ 14.223317] 0000001075ec7000 00000000013b7000 0000000001080000 0000000000000000
> (XEN) [ 14.226178] ffff830000000002 ffff830000097c60 ffff830000097ee0 ffff830000097fb0
> (XEN) [ 14.229039] 0000000000000000 0000001100000000 0000000800000000 000000010000006e
> (XEN) [ 14.231901] 0000000000000003 00000000000002f8 0000000000000000 0000000000000000
> (XEN) [ 14.234770] 00000000000000e8 0000000068924a50 000000006c5b8a28 000000006c5b8a40
> (XEN) [ 14.237640] 0000000000000000 ffff82d0802000f3 0000000000000000 0000000000000000
> (XEN) [ 14.240504] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) [ 14.243374] 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) [ 14.246245] Xen call trace:
> (XEN) [ 14.247668] [<ffff82d0802ef735>] vioapic_init+0x110/0x167
> (XEN) [ 14.249126] [<ffff82d0802de276>] hvm_domain_initialise+0x268/0x365
> (XEN) [ 14.250593] [<ffff82d08026d593>] arch_domain_create+0x4b0/0x6ac
> (XEN) [ 14.252055] [<ffff82d080205226>] domain_create+0x3d8/0x593
> (XEN) [ 14.253578] [<ffff82d0803ee1be>] __start_xen+0x20fa/0x24c3
> (XEN) [ 14.255029] [<ffff82d0802000f3>] __high_start+0x53/0x58
> (XEN) [ 14.256481]
> (XEN) [ 14.510271]
> (XEN) [ 14.513527] ****************************************
> (XEN) [ 14.520279] Panic on CPU 0:
> (XEN) [ 14.524726] Assertion 'hvm_domain_irq(d)->nr_gsis == nr_gsis' failed at vioapic.c:600
> (XEN) [ 14.535470] ****************************************
> (XEN) [ 14.542091]
> (XEN) [ 14.545230] Reboot in five seconds...
> (XEN) [ 19.553630] Resetting with ACPI MEMORY or I/O RESET_REG.
This is certainly unexpected. Can you try again with the debug patch below
applied? It should show what's actually wrong.
Thanks, Roger.
---8<---
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 527ac2aadd..1df7710041 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -625,6 +625,9 @@ int vioapic_init(struct domain *d)
nr_gsis += nr_pins;
}
+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
+
ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
d->arch.hvm_domain.nr_vioapics = nr_vioapics;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 1:03 ` Chao Gao
@ 2017-04-17 8:38 ` Roger Pau Monné
2017-04-17 2:49 ` Chao Gao
0 siblings, 1 reply; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-17 8:38 UTC (permalink / raw)
To: xen-devel, Kevin Tian
On Mon, Apr 17, 2017 at 09:03:12AM +0800, Chao Gao wrote:
> On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote:
> >On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
> >> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
> >> >Hello,
> >> >
> >> >Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
> >> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
> >> >completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
> >> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
> >> >
> >> >I'm not able to debug that in any meaningful way because the box seems to lock
> >> >up completely, even the watchdog NMI stops working. Here is the boot log, up to
> >> >the point where it freezes:
> >>
> >> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
> >>
> >---8<---
> >diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
> >index 527ac2aadd..1df7710041 100644
> >--- a/xen/arch/x86/hvm/vioapic.c
> >+++ b/xen/arch/x86/hvm/vioapic.c
> >@@ -625,6 +625,9 @@ int vioapic_init(struct domain *d)
> > nr_gsis += nr_pins;
> > }
> >
> >+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
> >+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
> >+
> > ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
> >
> > d->arch.hvm_domain.nr_vioapics = nr_vioapics;
>
> With the above patch,
> (XEN) [ 10.420001] PCI: MCFG area at 80000000 reserved in E820
> (XEN) [ 10.426854] PCI: Using MCFG for segment 0000 bus 00-ff
> (XEN) [ 10.433952] Intel VT-d iommu 6 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.441856] Intel VT-d iommu 5 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.449759] Intel VT-d iommu 4 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.457671] Intel VT-d iommu 3 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.465585] Intel VT-d iommu 2 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.473485] Intel VT-d iommu 1 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.481394] Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.489299] Intel VT-d iommu 7 supported page sizes: 4kB, 2MB, 1GB.
> (XEN) [ 10.497196] Intel VT-d Snoop Control enabled.
> (XEN) [ 10.503196] Intel VT-d Dom0 DMA Passthrough not enabled.
> (XEN) [ 10.510145] Intel VT-d Queued Invalidation enabled.
> (XEN) [ 10.516646] Intel VT-d Interrupt Remapping enabled.
> (XEN) [ 10.523173] Intel VT-d Posted Interrupt not enabled.
> (XEN) [ 10.529775] Intel VT-d Shared EPT tables enabled.
> (XEN) [ 10.548815] I/O virtualisation enabled
> (XEN) [ 10.554186] - Dom0 mode: Relaxed
> (XEN) [ 10.559264] Interrupt remapping enabled
> (XEN) [ 10.564854] nr_sockets: 5
> (XEN) [ 10.569231] Enabled directed EOI with ioapic_ack_old on!
> (XEN) [ 10.577294] ENABLING IO-APIC IRQs
> (XEN) [ 10.582245] -> Using old ACK method
> (XEN) [ 10.587967] ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) [ 10.797645] TSC deadline timer enabled
> (XEN) [ 10.887286] Defaulting to alternative key handling; send 'A' to switch to normal mode.
> (XEN) [ 10.897864] mwait-idle: MWAIT substates: 0x2020
> (XEN) [ 10.899335] mwait-idle: v0.4.1 model 0x55
> (XEN) [ 10.900799] mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) [ 10.902304] VMX: Supported advanced features:
> (XEN) [ 10.903781] - APIC MMIO access virtualisation
> (XEN) [ 10.905258] - APIC TPR shadow
> (XEN) [ 10.907138] - Extended Page Tables (EPT)
> (XEN) [ 10.908782] - Virtual-Processor Identifiers (VPID)
> (XEN) [ 10.910262] - Virtual NMI
> (XEN) [ 10.911719] - MSR direct-access bitmap
> (XEN) [ 10.913188] - Unrestricted Guest
> (XEN) [ 10.914650] - APIC Register Virtualization
> (XEN) [ 10.916126] - Virtual Interrupt Delivery
> (XEN) [ 10.917596] - Posted Interrupt Processing
> (XEN) [ 10.919066] - VMCS shadowing
> (XEN) [ 10.920519] - VM Functions
> (XEN) [ 10.921976] - Virtualisation Exceptions
> (XEN) [ 10.923448] - Page Modification Logging
> (XEN) [ 10.924918] - TSC Scaling
> (XEN) [ 10.926371] HVM: ASIDs enabled.
> (XEN) [ 10.927829] HVM: VMX enabled
> (XEN) [ 10.929278] HVM: Hardware Assisted Paging (HAP) detected
> (XEN) [ 10.930762] HVM: HAP page sizes: 4kB, 2MB, 1GB
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 6, using 0x1
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 9, using 0x1
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 10, using 0x1
> (XEN) [ 0.000000] CMCI: threshold 0x2 too large for CPU56 bank 11, using 0x1
> (XEN) [ 13.216648] Brought up 112 CPUs
> (XEN) [ 13.739330] build-id: dc4540250abe5d96614d340c67069e390c37c21c
> (XEN) [ 13.740816] Running stub recovery selftests...
> (XEN) [ 13.742258] traps.c:3466: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080359cf2
> (XEN) [ 13.745155] traps.c:813: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] -> ffff82d080359cf2
> (XEN) [ 13.748046] traps.c:1215: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d080359cf2
> (XEN) [ 13.771452] TSC warp detected, disabling TSC_RELIABLE
> (XEN) [ 13.772897] ACPI sleep modes: S3
> (XEN) [ 13.774329] VPMU: disabled
> (XEN) [ 13.775757] mcheck_poll: Machine check polling timer started.
> (XEN) [ 13.777246] Dom0 has maximum 1448 PIRQs
> (XEN) [ 13.779426] domain nr_gsis: 104 vioapic gsis: 88 nr_irqs_gsi: 104 highest_gsi: 103
It seems like we are not properly adding/accounting the vIO APICs, but I cannot
really see how. I have another patch for you to try below.
Thanks, Roger.
---8<---
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 527ac2aadd..40075e2756 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -610,11 +610,15 @@ int vioapic_init(struct domain *d)
xzalloc_array(struct hvm_vioapic *, nr_vioapics)) == NULL) )
return -ENOMEM;
+ printk("Adding %u vIO APICs\n", nr_vioapics);
+
for ( i = 0; i < nr_vioapics; i++ )
{
unsigned int nr_pins = is_hardware_domain(d) ? nr_ioapic_entries[i] :
ARRAY_SIZE(domain_vioapic(d, 0)->domU.redirtbl);
+ printk("vIO APIC %u has %u pins\n", i, nr_pins);
+
if ( (domain_vioapic(d, i) =
xmalloc_bytes(hvm_vioapic_size(nr_pins))) == NULL )
{
@@ -623,8 +627,12 @@ int vioapic_init(struct domain *d)
}
domain_vioapic(d, i)->nr_pins = nr_pins;
nr_gsis += nr_pins;
+ printk("nr_gsis: %u\n", nr_gsis);
}
+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
+
ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
d->arch.hvm_domain.nr_vioapics = nr_vioapics;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 2:49 ` Chao Gao
@ 2017-04-17 10:38 ` Roger Pau Monné
2017-04-17 5:12 ` Chao Gao
0 siblings, 1 reply; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-17 10:38 UTC (permalink / raw)
To: xen-devel, Kevin Tian, Chao Gao
On Mon, Apr 17, 2017 at 10:49:45AM +0800, Chao Gao wrote:
> On Mon, Apr 17, 2017 at 09:38:54AM +0100, Roger Pau Monné wrote:
> >On Mon, Apr 17, 2017 at 09:03:12AM +0800, Chao Gao wrote:
> >> On Mon, Apr 17, 2017 at 08:47:48AM +0100, Roger Pau Monné wrote:
> >> >On Mon, Apr 17, 2017 at 07:32:45AM +0800, Chao Gao wrote:
> >> >> On Fri, Apr 14, 2017 at 04:34:41PM +0100, Roger Pau Monné wrote:
> >> >> >Hello,
> >> >> >
> >> >> >Although PVHv2 Dom0 is not yet finished, I've been trying the current code on
> >> >> >different hardware, and found that with pre-Haswell Intel hardware PVHv2 Dom0
> >> >> >completely freezes the box when calling iommu_hwdom_init in dom0_construct_pvh.
> >> >> >OTOH the same doesn't happen when using a newer CPU (ie: haswell or newer).
> >> >> >
> >> >> >I'm not able to debug that in any meaningful way because the box seems to lock
> >> >> >up completely, even the watchdog NMI stops working. Here is the boot log, up to
> >> >> >the point where it freezes:
> >> >>
> >> >> I try "dom0=pvh" with my skylake. An assertion failed. Is it a software bug?
> >> >>
> >
> >It seems like we are not properly adding/accounting the vIO APICs, but I cannot
> >really see how. I have another patch for you to try below.
> >
> >Thanks, Roger.
> >
> >---8<---
> > diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
> >index 527ac2aadd..40075e2756 100644
> >--- a/xen/arch/x86/hvm/vioapic.c
> >+++ b/xen/arch/x86/hvm/vioapic.c
> >@@ -610,11 +610,15 @@ int vioapic_init(struct domain *d)
> > xzalloc_array(struct hvm_vioapic *, nr_vioapics)) == NULL) )
> > return -ENOMEM;
> >
> >+ printk("Adding %u vIO APICs\n", nr_vioapics);
> >+
> > for ( i = 0; i < nr_vioapics; i++ )
> > {
> > unsigned int nr_pins = is_hardware_domain(d) ? nr_ioapic_entries[i] :
> > ARRAY_SIZE(domain_vioapic(d, 0)->domU.redirtbl);
> >
> >+ printk("vIO APIC %u has %u pins\n", i, nr_pins);
> >+
> > if ( (domain_vioapic(d, i) =
> > xmalloc_bytes(hvm_vioapic_size(nr_pins))) == NULL )
> > {
> >@@ -623,8 +627,12 @@ int vioapic_init(struct domain *d)
> > }
> > domain_vioapic(d, i)->nr_pins = nr_pins;
> > nr_gsis += nr_pins;
> >+ printk("nr_gsis: %u\n", nr_gsis);
> > }
> >
> >+ printk("domain nr_gsis: %u vioapic gsis: %u nr_irqs_gsi: %u highest_gsi: %u\n",
> >+ hvm_domain_irq(d)->nr_gsis, nr_gsis, nr_irqs_gsi, highest_gsi());
> >+
> > ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
> >
> > d->arch.hvm_domain.nr_vioapics = nr_vioapics;
> >
>
> Please Cc or To me. Is there some holes in all physical IOAPICs gsi ranges?
That's weird, my MUA (Mutt) seems to automatically remove your address from the
"To:" field. I have no idea why it does that.
So yes, your box has as GSI gap which is not handled by any IO APIC. TBH, I
didn't even knew that was possible. In any case, patch below should solve it.
---8<---
commit f52d05fca03440d771eb56077c9d60bb630eb423
diff --git a/xen/arch/x86/hvm/vioapic.c b/xen/arch/x86/hvm/vioapic.c
index 5157db7a4e..ec87a97651 100644
--- a/xen/arch/x86/hvm/vioapic.c
+++ b/xen/arch/x86/hvm/vioapic.c
@@ -64,37 +64,23 @@ static struct hvm_vioapic *addr_vioapic(const struct domain *d,
struct hvm_vioapic *gsi_vioapic(const struct domain *d, unsigned int gsi,
unsigned int *pin)
{
- unsigned int i, base_gsi = 0;
+ unsigned int i;
for ( i = 0; i < d->arch.hvm_domain.nr_vioapics; i++ )
{
struct hvm_vioapic *vioapic = domain_vioapic(d, i);
- if ( gsi >= base_gsi && gsi < base_gsi + vioapic->nr_pins )
+ if ( gsi >= vioapic->base_gsi &&
+ gsi < vioapic->base_gsi + vioapic->nr_pins )
{
- *pin = gsi - base_gsi;
+ *pin = gsi - vioapic->base_gsi;
return vioapic;
}
-
- base_gsi += vioapic->nr_pins;
}
return NULL;
}
-static unsigned int base_gsi(const struct domain *d,
- const struct hvm_vioapic *vioapic)
-{
- unsigned int nr_vioapics = d->arch.hvm_domain.nr_vioapics;
- unsigned int base_gsi = 0, i = 0;
- const struct hvm_vioapic *tmp;
-
- while ( i < nr_vioapics && (tmp = domain_vioapic(d, i++)) != vioapic )
- base_gsi += tmp->nr_pins;
-
- return base_gsi;
-}
-
static uint32_t vioapic_read_indirect(const struct hvm_vioapic *vioapic)
{
uint32_t result = 0;
@@ -180,7 +166,7 @@ static void vioapic_write_redirent(
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
union vioapic_redir_entry *pent, ent;
int unmasked = 0;
- unsigned int gsi = base_gsi(d, vioapic) + idx;
+ unsigned int gsi = vioapic->base_gsi + idx;
spin_lock(&d->arch.hvm_domain.irq_lock);
@@ -340,7 +326,7 @@ static void vioapic_deliver(struct hvm_vioapic *vioapic, unsigned int pin)
struct domain *d = vioapic_domain(vioapic);
struct vlapic *target;
struct vcpu *v;
- unsigned int irq = base_gsi(d, vioapic) + pin;
+ unsigned int irq = vioapic->base_gsi + pin;
ASSERT(spin_is_locked(&d->arch.hvm_domain.irq_lock));
@@ -451,7 +437,7 @@ void vioapic_update_EOI(struct domain *d, u8 vector)
{
struct hvm_irq *hvm_irq = hvm_domain_irq(d);
union vioapic_redir_entry *ent;
- unsigned int i, base_gsi = 0;
+ unsigned int i;
ASSERT(has_vioapic(d));
@@ -473,19 +459,18 @@ void vioapic_update_EOI(struct domain *d, u8 vector)
if ( iommu_enabled )
{
spin_unlock(&d->arch.hvm_domain.irq_lock);
- hvm_dpci_eoi(d, base_gsi + pin, ent);
+ hvm_dpci_eoi(d, vioapic->base_gsi + pin, ent);
spin_lock(&d->arch.hvm_domain.irq_lock);
}
if ( (ent->fields.trig_mode == VIOAPIC_LEVEL_TRIG) &&
!ent->fields.mask &&
- hvm_irq->gsi_assert_count[base_gsi + pin] )
+ hvm_irq->gsi_assert_count[vioapic->base_gsi + pin] )
{
ent->fields.remote_irr = 1;
vioapic_deliver(vioapic, pin);
}
}
- base_gsi += vioapic->nr_pins;
}
spin_unlock(&d->arch.hvm_domain.irq_lock);
@@ -554,6 +539,7 @@ void vioapic_reset(struct domain *d)
{
vioapic->base_address = mp_ioapics[i].mpc_apicaddr;
vioapic->id = mp_ioapics[i].mpc_apicid;
+ vioapic->base_gsi = io_apic_gsi_base(i);
}
vioapic->nr_pins = nr_pins;
vioapic->domain = d;
@@ -601,7 +587,12 @@ int vioapic_init(struct domain *d)
nr_gsis += nr_pins;
}
- ASSERT(hvm_domain_irq(d)->nr_gsis == nr_gsis);
+ /*
+ * NB: hvm_domain_irq(d)->nr_gsis is actually the highest GSI + 1, but
+ * there might be holes in this range (ie: GSIs that don't belong to any
+ * vIO APIC).
+ */
+ ASSERT(hvm_domain_irq(d)->nr_gsis >= nr_gsis);
d->arch.hvm_domain.nr_vioapics = nr_vioapics;
vioapic_reset(d);
diff --git a/xen/include/asm-x86/hvm/vioapic.h b/xen/include/asm-x86/hvm/vioapic.h
index 8ec91d2bd1..2ceb60eaef 100644
--- a/xen/include/asm-x86/hvm/vioapic.h
+++ b/xen/include/asm-x86/hvm/vioapic.h
@@ -50,6 +50,7 @@
struct hvm_vioapic {
struct domain *domain;
uint32_t nr_pins;
+ unsigned int base_gsi;
union {
XEN_HVM_VIOAPIC(,);
struct hvm_hw_vioapic domU;
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 5:12 ` Chao Gao
@ 2017-04-17 12:21 ` Roger Pau Monné
2017-04-17 5:47 ` Chao Gao
0 siblings, 1 reply; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-17 12:21 UTC (permalink / raw)
To: xen-devel, Kevin Tian, Chao Gao
On Mon, Apr 17, 2017 at 01:12:27PM +0800, Chao Gao wrote:
[...]
> It works. I can test for you when you send out a formal patch.
Thanks for the testing, will send formal patch shortly.
Have you been able to reproduce the IOMMU issue with that, or you just hit the
panic at the end of PVH Dom0 build?
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-17 5:47 ` Chao Gao
@ 2017-04-17 12:57 ` Roger Pau Monné
2017-04-17 6:18 ` Chao Gao
0 siblings, 1 reply; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-17 12:57 UTC (permalink / raw)
To: xen-devel, Kevin Tian, Chao Gao
On Mon, Apr 17, 2017 at 01:47:47PM +0800, Chao Gao wrote:
> On Mon, Apr 17, 2017 at 01:21:01PM +0100, Roger Pau Monné wrote:
> >On Mon, Apr 17, 2017 at 01:12:27PM +0800, Chao Gao wrote:
> >[...]
> >> It works. I can test for you when you send out a formal patch.
> >
> >Thanks for the testing, will send formal patch shortly.
> >
> >Have you been able to reproduce the IOMMU issue with that, or you just hit the
> >panic at the end of PVH Dom0 build?
>
> No, I haven't. The output is some like ELFxxx not found, I think, due to lack
> of pvh domain0 kernel. As mentioned before, my platform is skylake.
Right, if you get to the ELF stuff it means the IOMMU has been initialized
successfully. Skylake is post-haswell, so I don't think it's going to exhibit
those issues. Is there any chance you can test on something older
(pre-haswell?).
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-14 15:34 PVH Dom0 Intel IOMMU issues Roger Pau Monné
2017-04-16 23:32 ` Chao Gao
@ 2017-04-18 3:04 ` Tian, Kevin
2017-04-18 7:34 ` Roger Pau Monné
1 sibling, 1 reply; 16+ messages in thread
From: Tian, Kevin @ 2017-04-18 3:04 UTC (permalink / raw)
To: Roger Pau Monné, xen-devel
> From: Roger Pau Monné [mailto:roger.pau@citrix.com]
> Sent: Friday, April 14, 2017 11:35 PM
>
> Hello,
>
> Although PVHv2 Dom0 is not yet finished, I've been trying the current code
> on
> different hardware, and found that with pre-Haswell Intel hardware PVHv2
> Dom0
> completely freezes the box when calling iommu_hwdom_init in
> dom0_construct_pvh.
> OTOH the same doesn't happen when using a newer CPU (ie: haswell or
> newer).
>
> I'm not able to debug that in any meaningful way because the box seems to
> lock
> up completely, even the watchdog NMI stops working. Here is the boot log,
> up to
> the point where it freezes:
>
I don't have an idea now w/o seeing more meaningful debug message.
Maybe you have to add more fine-grained prints to capture some
useful hints.
Thanks
Kevin
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-18 3:04 ` Tian, Kevin
@ 2017-04-18 7:34 ` Roger Pau Monné
2017-04-18 8:48 ` Jan Beulich
0 siblings, 1 reply; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-18 7:34 UTC (permalink / raw)
To: Tian, Kevin; +Cc: xen-devel
On Tue, Apr 18, 2017 at 03:04:51AM +0000, Tian, Kevin wrote:
> > From: Roger Pau Monné [mailto:roger.pau@citrix.com]
> > Sent: Friday, April 14, 2017 11:35 PM
> >
> > Hello,
> >
> > Although PVHv2 Dom0 is not yet finished, I've been trying the current code
> > on
> > different hardware, and found that with pre-Haswell Intel hardware PVHv2
> > Dom0
> > completely freezes the box when calling iommu_hwdom_init in
> > dom0_construct_pvh.
> > OTOH the same doesn't happen when using a newer CPU (ie: haswell or
> > newer).
> >
> > I'm not able to debug that in any meaningful way because the box seems to
> > lock
> > up completely, even the watchdog NMI stops working. Here is the boot log,
> > up to
> > the point where it freezes:
> >
>
> I don't have an idea now w/o seeing more meaningful debug message.
> Maybe you have to add more fine-grained prints to capture some
> useful hints.
Hello, I've added the following debug patch:
diff --git a/xen/drivers/passthrough/vtd/iommu.c b/xen/drivers/passthrough/vtd/iommu.c
index a5c61c6e21..cb039d74e7 100644
--- a/xen/drivers/passthrough/vtd/iommu.c
+++ b/xen/drivers/passthrough/vtd/iommu.c
@@ -765,7 +765,9 @@ static void iommu_enable_translation(struct acpi_drhd_unit *drhd)
iommu->reg);
spin_lock_irqsave(&iommu->register_lock, flags);
sts = dmar_readl(iommu->reg, DMAR_GSTS_REG);
+ printk("Before DMA_GCMD_TE\n");
dmar_writel(iommu->reg, DMAR_GCMD_REG, sts | DMA_GCMD_TE);
+ printk("After DMA_GCMD_TE\n");
/* Make sure hardware complete it */
IOMMU_WAIT_OP(iommu, DMAR_GSTS_REG, dmar_readl,
And got this output:
(XEN) Xen version 4.9-rc (root@) (FreeBSD clang version 3.9.0 (tags/RELEASE_390/final 280324) (based on LLVM 3.9.0)) debug=y Tue Apr 18 08:22:39 BST 2017
(XEN) Latest ChangeSet:
(XEN) Console output is synchronous.
(XEN) Bootloader: FreeBSD Loader
(XEN) Command line: dom0_mem=4096M dom0=pvh com1=115200,8n1 console=com1,vga guest_loglvl=all loglvl=all iommu=debug,verbose sync_console watchdog
(XEN) Xen image load base address: 0
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN) Found 2 MBR signatures
(XEN) Found 2 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000008dc00 (usable)
(XEN) 000000000008dc00 - 00000000000a0000 (reserved)
(XEN) 00000000000e0000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 0000000018ebb000 (usable)
(XEN) 0000000018ebb000 - 0000000018fe8000 (ACPI NVS)
(XEN) 0000000018fe8000 - 0000000018fe9000 (usable)
(XEN) 0000000018fe9000 - 0000000019000000 (ACPI NVS)
(XEN) 0000000019000000 - 000000001dffd000 (usable)
(XEN) 000000001dffd000 - 000000001e000000 (ACPI data)
(XEN) 000000001e000000 - 00000000ac784000 (usable)
(XEN) 00000000ac784000 - 00000000ac818000 (reserved)
(XEN) 00000000ac818000 - 00000000ad800000 (usable)
(XEN) 00000000b0000000 - 00000000b4000000 (reserved)
(XEN) 00000000fed20000 - 00000000fed40000 (reserved)
(XEN) 00000000fed50000 - 00000000fed90000 (reserved)
(XEN) 00000000ffa00000 - 00000000ffa40000 (reserved)
(XEN) 0000000100000000 - 0000000250000000 (usable)
(XEN) New Xen image base address: 0xad200000
(XEN) ACPI: RSDP 000FE300, 0024 (r2 DELL )
(XEN) ACPI: XSDT 1DFFEE18, 0074 (r1 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: FACP 18FEFD98, 00F4 (r4 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: DSDT 18FA9018, 6373 (r1 DELL CBX3 0 INTL 20091112)
(XEN) ACPI: FACS 18FF1F40, 0040
(XEN) ACPI: APIC 1DFFDC18, 0158 (r2 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: MCFG 18FFED18, 003C (r1 A M I OEMMCFG. 6222004 MSFT 97)
(XEN) ACPI: TCPA 18FFEC98, 0032 (r2 0 0)
(XEN) ACPI: SSDT 18FF0A98, 0306 (r1 DELLTP TPM 3000 INTL 20091112)
(XEN) ACPI: HPET 18FFEC18, 0038 (r1 A M I PCHHPET 6222004 AMI. 3)
(XEN) ACPI: BOOT 18FFEB98, 0028 (r1 DELL CBX3 6222004 AMI 10013)
(XEN) ACPI: SSDT 18FB0018, 36FFE (r2 INTEL CpuPm 4000 INTL 20091112)
(XEN) ACPI: SLIC 18FEEC18, 0176 (r3 DELL CBX3 6222004 MSFT 10013)
(XEN) ACPI: DMAR 18FF1B18, 0094 (r1 A M I OEMDMAR 1 INTL 1)
(XEN) System RAM: 8149MB (8345288kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at 0000000000000000-0000000250000000
(XEN) Domain heap initialised
(XEN) CPU Vendor: Intel, Family 6 (0x6), Model 45 (0x2d), Stepping 7 (raw 000206d7)
(XEN) found SMP MP-table at 000f1db0
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x408 (32 bits)
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:404,1:0], pm1x_evt[1:400,1:0]
(XEN) ACPI: 32/64X FACS address mismatch in FADT - 18ffdf40/0000000018ff1f40, using 32
(XEN) ACPI: wakeup_vec[18ffdf4c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x04] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x05] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x06] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x09] lapic_id[0x08] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0a] lapic_id[0x09] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0b] lapic_id[0x0a] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0c] lapic_id[0x0b] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0d] lapic_id[0x0c] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0e] lapic_id[0x0d] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x0f] lapic_id[0x0e] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x10] lapic_id[0x0f] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x11] lapic_id[0x10] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x12] lapic_id[0x11] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x13] lapic_id[0x12] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x14] lapic_id[0x13] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x15] lapic_id[0x14] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x16] lapic_id[0x15] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x17] lapic_id[0x16] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x18] lapic_id[0x17] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x19] lapic_id[0x18] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1a] lapic_id[0x19] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1b] lapic_id[0x1a] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1c] lapic_id[0x1b] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1d] lapic_id[0x1c] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1e] lapic_id[0x1d] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x1f] lapic_id[0x1e] disabled)
(XEN) ACPI: LAPIC (acpi_id[0x20] lapic_id[0x1f] disabled)
(XEN) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23
(XEN) ACPI: IOAPIC (id[0x02] address[0xfec3f000] gsi_base[24])
(XEN) IOAPIC[1]: apic_id 2, version 32, address 0xfec3f000, GSI 24-47
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode: Flat. Using 2 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed00000
(XEN) [VT-D]Host address width 46
(XEN) [VT-D]found ACPI_DMAR_DRHD:
(XEN) [VT-D] dmaru->address = fbffe000
(XEN) [VT-D]drhd->address = fbffe000 iommu->reg = ffff82c00021b000
(XEN) [VT-D]cap = d2078c106f0462 ecap = f020fa
(XEN) [VT-D] IOAPIC: 0000:00:1f.7
(XEN) [VT-D] IOAPIC: 0000:00:05.4
(XEN) [VT-D] MSI HPET: 0000:f0:0f.0
(XEN) [VT-D] flags: INCLUDE_ALL
(XEN) [VT-D]found ACPI_DMAR_RMRR:
(XEN) [VT-D] endpoint: 0000:00:1d.0
(XEN) [VT-D] endpoint: 0000:00:1a.0
(XEN) [VT-D]dmar.c:638: RMRR region: base_addr ac7cf000 end_addr ac7defff
(XEN) [VT-D]found ACPI_DMAR_RHSA:
(XEN) [VT-D] rhsau->address: fbffe000 rhsau->proximity_domain: 0
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 32 CPUs (28 hotplug CPUs)
(XEN) IRQ limits: 48 GSI, 736 MSI/MSI-X
(XEN) Switched to APIC driver x2apic_cluster.
(XEN) xstate: size: 0x340 and states: 0x7
(XEN) mce_intel.c:732: MCA capability: firstbank 0, 0 ext MSRs, BCAST, SER, CMCI
(XEN) CPU0: Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Platform timer is 14.318MHz HPET
(XEN) Detected 2992.792 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0804181f0 -> ffff82d080418964
(XEN) PCI: MCFG configuration 0: base b0000000 segment 0000 buses 00 - 3f
(XEN) PCI: MCFG area at b0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB, 2MB, 1GB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-d Queued Invalidation enabled.
(XEN) Intel VT-d Interrupt Remapping enabled.
(XEN) Intel VT-d Posted Interrupt not enabled.
(XEN) Intel VT-d Shared EPT tables enabled.
(XEN) I/O virtualisation enabled
(XEN) - Dom0 mode: Relaxed
(XEN) Interrupt remapping enabled
(XEN) nr_sockets: 9
(XEN) Enabled directed EOI with ioapic_ack_old on!
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using old ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) TSC deadline timer enabled
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x21120
(XEN) mwait-idle: v0.4.1 model 0x2d
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC MMIO access virtualisation
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) detected
(XEN) HVM: HAP page sizes: 4kB, 2MB, 1GB
(XEN) Brought up 4 CPUs
(XEN) Testing NMI watchdog on all CPUs: ok
(XEN) Running stub recovery selftests...
(XEN) traps.c:3457: GPF (0000): ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d08034295e
(XEN) traps.c:813: Trap 12: ffff82d0bffff040 [ffff82d0bffff040] -> ffff82d08034295e
(XEN) traps.c:1215: Trap 3: ffff82d0bffff041 [ffff82d0bffff041] -> ffff82d08034295e
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) mcheck_poll: Machine check polling timer started.
(XEN) Dom0 has maximum 624 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) ** Building a PVH Dom0 **
(XEN) [VT-D]d0:Hostbridge: skip 0000:00:00.0 map
(XEN) Masked UR signaling on 0000:00:00.0
(XEN) Masked UR signaling on 0000:00:01.0
(XEN) Masked UR signaling on 0000:00:01.1
(XEN) Masked UR signaling on 0000:00:02.0
(XEN) Masked UR signaling on 0000:00:03.0
(XEN) [VT-D]d0:PCIe: map 0000:00:05.0
(XEN) Masked VT-d error signaling on 0000:00:05.0
(XEN) [VT-D]d0:PCIe: map 0000:00:05.2
(XEN) [VT-D]d0:PCI: map 0000:00:05.4
(XEN) [VT-D]d0:PCI: map 0000:00:16.0
(XEN) [VT-D]d0:PCI: map 0000:00:19.0
(XEN) [VT-D]d0:PCI: map 0000:00:1a.0
(XEN) [VT-D]d0:PCIe: map 0000:00:1b.0
(XEN) [VT-D]d0:PCI: map 0000:00:1d.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.0
(XEN) [VT-D]d0:PCI: map 0000:00:1f.2
(XEN) [VT-D]d0:PCI: map 0000:00:1f.3
(XEN) [VT-D]d0:PCIe: map 0000:03:00.0
(XEN) [VT-D]d0:PCIe: map 0000:03:00.1
(XEN) [VT-D]d0:PCIe: map 0000:05:00.0
(XEN) [VT-D]d0:PCIe: map 0000:05:00.3
(XEN) [VT-D]d0:PCIe: map 0000:07:00.0
(XEN) [VT-D]d0:PCI: map 0000:3f:08.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:08.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:08.4
(XEN) [VT-D]d0:PCI: map 0000:3f:09.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:09.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:09.4
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.1
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.2
(XEN) [VT-D]d0:PCI: map 0000:3f:0a.3
(XEN) [VT-D]d0:PCI: map 0000:3f:0b.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0b.3
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.1
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.6
(XEN) [VT-D]d0:PCI: map 0000:3f:0c.7
(XEN) [VT-D]d0:PCI: map 0000:3f:0d.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0d.1
(XEN) [VT-D]d0:PCI: map 0000:3f:0d.6
(XEN) [VT-D]d0:PCI: map 0000:3f:0e.0
(XEN) [VT-D]d0:PCI: map 0000:3f:0e.1
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.1
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.2
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.4
(XEN) [VT-D]d0:PCIe: map 0000:3f:0f.5
(XEN) [VT-D]d0:PCI: map 0000:3f:0f.6
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.0
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.1
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.2
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.3
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.4
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.5
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.6
(XEN) [VT-D]d0:PCIe: map 0000:3f:10.7
(XEN) [VT-D]d0:PCI: map 0000:3f:11.0
(XEN) [VT-D]d0:PCI: map 0000:3f:13.0
(XEN) [VT-D]d0:PCI: map 0000:3f:13.1
(XEN) [VT-D]d0:PCI: map 0000:3f:13.4
(XEN) [VT-D]d0:PCI: map 0000:3f:13.5
(XEN) [VT-D]d0:PCI: map 0000:3f:13.6
(XEN) [VT-D]iommu_enable_translation: iommu->reg = ffff82c00021b000
(XEN) Before DMA_GCMD_TE
(
The hang seems to happen when writing DMA_GCMD_TE to the global command
register, which enables the DMA remapping. After that the box is completely
unresponsive, not even the watchdog is working.
Thanks, Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-18 7:34 ` Roger Pau Monné
@ 2017-04-18 8:48 ` Jan Beulich
2017-04-18 8:59 ` Roger Pau Monné
0 siblings, 1 reply; 16+ messages in thread
From: Jan Beulich @ 2017-04-18 8:48 UTC (permalink / raw)
To: Roger Pau Monné; +Cc: xen-devel, Kevin Tian
>>> On 18.04.17 at 09:34, <roger.pau@citrix.com> wrote:
> (XEN) Before DMA_GCMD_TE
> (
>
> The hang seems to happen when writing DMA_GCMD_TE to the global command
> register, which enables the DMA remapping. After that the box is completely
> unresponsive, not even the watchdog is working.
How sure are you that this is pre-Haswell specific vs e.g. chipset or
firmware (think of RMRRs [or their lack] for the latter) dependent?
Iirc Elena's command line specifiable RMRR patch series was
motivated by similar behavior she had observed on some system.
Another odd aspect is - why would IOMMU enabling cause the hang
only when intending to use a PVH Dom0? The IOMMU is being
enabled in either case, which again might point at differences in use
of memory.
Jan
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: PVH Dom0 Intel IOMMU issues
2017-04-18 8:48 ` Jan Beulich
@ 2017-04-18 8:59 ` Roger Pau Monné
0 siblings, 0 replies; 16+ messages in thread
From: Roger Pau Monné @ 2017-04-18 8:59 UTC (permalink / raw)
To: Jan Beulich; +Cc: xen-devel, Kevin Tian
On Tue, Apr 18, 2017 at 02:48:31AM -0600, Jan Beulich wrote:
> >>> On 18.04.17 at 09:34, <roger.pau@citrix.com> wrote:
> > (XEN) Before DMA_GCMD_TE
> > (
> >
> > The hang seems to happen when writing DMA_GCMD_TE to the global command
> > register, which enables the DMA remapping. After that the box is completely
> > unresponsive, not even the watchdog is working.
>
> How sure are you that this is pre-Haswell specific vs e.g. chipset or
> firmware (think of RMRRs [or their lack] for the latter) dependent?
> Iirc Elena's command line specifiable RMRR patch series was
> motivated by similar behavior she had observed on some system.
This is mostly from trial/error. I don't think it's strictly CPU related, but
rather chipset related (ie: chipsets that come with pre-haswell CPUs).
Elena IIRC was at least getting IOMMU faults, which I don't even get in my
case, and I think that's the issue itself.
> Another odd aspect is - why would IOMMU enabling cause the hang
> only when intending to use a PVH Dom0? The IOMMU is being
> enabled in either case, which again might point at differences in use
> of memory.
Not sure, for PVH Dom0 the IOMMU is enabled quite early in the domain build
process (before populating the domain p2m), which seems to be fine on other
systems.
I've done that (initializing the IOMMU so early) to avoid having to iterate
over the list of domain pages afterwards when the IOMMU is initialized with the
p2m already populated.
FWIW, moving the iommu_hwdom_init call to the end of the PVH Dom0 build process
doesn't solve the issue. I've also tried with and without shared pt, and the
result is the same.
Roger.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2017-04-18 8:59 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-14 15:34 PVH Dom0 Intel IOMMU issues Roger Pau Monné
2017-04-16 23:32 ` Chao Gao
2017-04-17 7:47 ` Roger Pau Monné
2017-04-17 1:03 ` Chao Gao
2017-04-17 8:38 ` Roger Pau Monné
2017-04-17 2:49 ` Chao Gao
2017-04-17 10:38 ` Roger Pau Monné
2017-04-17 5:12 ` Chao Gao
2017-04-17 12:21 ` Roger Pau Monné
2017-04-17 5:47 ` Chao Gao
2017-04-17 12:57 ` Roger Pau Monné
2017-04-17 6:18 ` Chao Gao
2017-04-18 3:04 ` Tian, Kevin
2017-04-18 7:34 ` Roger Pau Monné
2017-04-18 8:48 ` Jan Beulich
2017-04-18 8:59 ` Roger Pau Monné
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.