* Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
@ 2015-11-20 1:22 Ed Swierk
2015-11-20 10:25 ` Jan Beulich
0 siblings, 1 reply; 22+ messages in thread
From: Ed Swierk @ 2015-11-20 1:22 UTC (permalink / raw)
To: xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 16872 bytes --]
Xen staging-4.6 crashes when booting on VMware Fusion 8.0.2 (with VT-x/EPT
enabled), with 4 virtual CPUs:
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=n Thu Nov 19
17:05:27 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M maxcpus=4 loglvl=all cpuinfo=1
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8fc000 - 1bd8fe000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.811 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802be010 -> ffff82d0802bf384
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 6 -> Node 0
(XEN) Booting processor 3/6 eip 8f000
(XEN) Initializing CPU#3
(XEN) CPU: Physical Processor ID: 6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) ----[ Xen-4.6.1-pre x86_64 debug=n Not tainted ]----
(XEN) CPU: 3
(XEN) RIP: e008:[<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
(XEN) RFLAGS: 0000000000010006 CONTEXT: hypervisor
(XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d5b4080
(XEN) rdx: 0000000000000006 rsi: 0000000000000000 rdi: 0000000000000003
(XEN) rbp: 0000000000000300 rsp: ffff8301bd87fe90 r8: ffff8301bd878000
(XEN) r9: 000000313d5b4080 r10: 0000000000000001 r11: 0000000000000001
(XEN) r12: ffff82d0802fd500 r13: 0000000000000000 r14: 0000000000000000
(XEN) r15: 0000000000000003 cr0: 000000008005003b cr4: 00000000001526a0
(XEN) cr3: 00000000bfc75000 cr2: 0000000000000001
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff8301bd87fe90:
(XEN) 00000003802fd800 0000000000000018 0000000000000000 0000010000000000
(XEN) ffff82d0802fd800 0000000000000000 00000000000000c8 0000000000000003
(XEN) 0000000000000000 0000000000000000 0000000000000000 ffff82d0801834dc
(XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000003 ffff8300bfafc000
(XEN) 000000313d5b4080 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
(XEN) [<ffff82d0801834dc>] start_secondary+0x1bc/0x250
(XEN)
(XEN) Pagetable walk from 0000000000000001:
(XEN) L4[0x000] = 00000001bd8f0063 ffffffffffffffff
(XEN) L3[0x000] = 00000001bd8ef063 ffffffffffffffff
(XEN) L2[0x000] = 00000001bd8ee063 ffffffffffffffff
(XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 3:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000001
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
set_cpu_sibling_map+0x3f is the second cpumask_set_cpu() call in
set_cpu_sibling_map():
http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/smpboot.c;h=094699286f4f6962942024ec8b2b24c7b7996cc0;hb=78833c04250416f1870c458309d3ac0e5cf915fd#l261
The crash does not occur if I set maxcpus=3 on the Xen command line:
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=n Thu Nov 19
17:05:27 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M maxcpus=3 loglvl=all cpuinfo=1
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8fc000 - 1bd8fe000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 3 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 568 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.652 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802be010 -> ffff82d0802bf384
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) Brought up 3 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) Dom0 has maximum 408 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
be allocated)
(XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
(XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
(XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
(XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
(XEN) Page tables: ffffffff8412d000->ffffffff84152000
(XEN) Boot stack: ffffffff84152000->ffffffff84153000
(XEN) TOTAL: ffffffff80000000->ffffffff84400000
(XEN) ENTRY ADDRESS: ffffffff81f991f0
(XEN) Dom0 has maximum 3 VCPUs
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 316kB init memory.
mapping kernel into physical memory
about to get started...
Actually I tried all numbers of CPUs between 2 and 8, and the crash occurs
only with 4 or 6 CPUs. Xen boots fine with 2, 3, 5, 7 or 8 CPUs.
Same results with the official 4.6.0 release.
Xen 4.5.1 boots fine on Fusion with any number of CPUs.
Any clues would be appreciated!
--Ed
[-- Attachment #1.2: Type: text/html, Size: 21276 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-20 1:22 Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion Ed Swierk
@ 2015-11-20 10:25 ` Jan Beulich
2015-11-20 10:37 ` Andrew Cooper
2015-11-20 18:53 ` Ed Swierk
0 siblings, 2 replies; 22+ messages in thread
From: Jan Beulich @ 2015-11-20 10:25 UTC (permalink / raw)
To: Ed Swierk; +Cc: xen-devel
>>> On 20.11.15 at 02:22, <eswierk@skyportsystems.com> wrote:
> (XEN) ----[ Xen-4.6.1-pre x86_64 debug=n Not tainted ]----
> (XEN) CPU: 3
> (XEN) RIP: e008:[<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
> (XEN) RFLAGS: 0000000000010006 CONTEXT: hypervisor
> (XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d5b4080
> (XEN) rdx: 0000000000000006 rsi: 0000000000000000 rdi: 0000000000000003
> (XEN) rbp: 0000000000000300 rsp: ffff8301bd87fe90 r8: ffff8301bd878000
> (XEN) r9: 000000313d5b4080 r10: 0000000000000001 r11: 0000000000000001
> (XEN) r12: ffff82d0802fd500 r13: 0000000000000000 r14: 0000000000000000
> (XEN) r15: 0000000000000003 cr0: 000000008005003b cr4: 00000000001526a0
> (XEN) cr3: 00000000bfc75000 cr2: 0000000000000001
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff8301bd87fe90:
> (XEN) 00000003802fd800 0000000000000018 0000000000000000 0000010000000000
> (XEN) ffff82d0802fd800 0000000000000000 00000000000000c8 0000000000000003
> (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff82d0801834dc
> (XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000003 ffff8300bfafc000
> (XEN) 000000313d5b4080 0000000000000000
> (XEN) Xen call trace:
> (XEN) [<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
> (XEN) [<ffff82d0801834dc>] start_secondary+0x1bc/0x250
> (XEN)
> (XEN) Pagetable walk from 0000000000000001:
> (XEN) L4[0x000] = 00000001bd8f0063 ffffffffffffffff
> (XEN) L3[0x000] = 00000001bd8ef063 ffffffffffffffff
> (XEN) L2[0x000] = 00000001bd8ee063 ffffffffffffffff
> (XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 3:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0002]
> (XEN) Faulting linear address: 0000000000000001
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
> set_cpu_sibling_map+0x3f is the second cpumask_set_cpu() call in
> set_cpu_sibling_map():
> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/smpboot.c;h=0
> 94699286f4f6962942024ec8b2b24c7b7996cc0;hb=78833c04250416f1870c458309d3ac0e5c
> f915fd#l261
I suppose cpu_to_socket(cpu) returns a value for which the
socket_cpumask[] entry didn't get set up yet. But to prove that,
we'd need to see the disassembly around the code location
above, to be able to associate register values with variables.
If that's the case, then I'd further guess that the CPUID
information provided by Fusion isn't exactly as one would expect
on real hardware. Whether we need to fix something, or can
work around a quirk of theirs depends on the exact nature of
the issue. Instrumenting code populating socket_cpumask[]
would be a good first step.
Jan
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-20 10:25 ` Jan Beulich
@ 2015-11-20 10:37 ` Andrew Cooper
2015-11-20 18:53 ` Ed Swierk
1 sibling, 0 replies; 22+ messages in thread
From: Andrew Cooper @ 2015-11-20 10:37 UTC (permalink / raw)
To: Jan Beulich, Ed Swierk; +Cc: xen-devel
On 20/11/15 10:25, Jan Beulich wrote:
>>>> On 20.11.15 at 02:22, <eswierk@skyportsystems.com> wrote:
>> (XEN) ----[ Xen-4.6.1-pre x86_64 debug=n Not tainted ]----
>> (XEN) CPU: 3
>> (XEN) RIP: e008:[<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
>> (XEN) RFLAGS: 0000000000010006 CONTEXT: hypervisor
>> (XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d5b4080
>> (XEN) rdx: 0000000000000006 rsi: 0000000000000000 rdi: 0000000000000003
>> (XEN) rbp: 0000000000000300 rsp: ffff8301bd87fe90 r8: ffff8301bd878000
>> (XEN) r9: 000000313d5b4080 r10: 0000000000000001 r11: 0000000000000001
>> (XEN) r12: ffff82d0802fd500 r13: 0000000000000000 r14: 0000000000000000
>> (XEN) r15: 0000000000000003 cr0: 000000008005003b cr4: 00000000001526a0
>> (XEN) cr3: 00000000bfc75000 cr2: 0000000000000001
>> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
>> (XEN) Xen stack trace from rsp=ffff8301bd87fe90:
>> (XEN) 00000003802fd800 0000000000000018 0000000000000000 0000010000000000
>> (XEN) ffff82d0802fd800 0000000000000000 00000000000000c8 0000000000000003
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 ffff82d0801834dc
>> (XEN) 0000000000000000 0000000000000001 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> (XEN) 0000000000000000 0000000000000000 0000000000000003 ffff8300bfafc000
>> (XEN) 000000313d5b4080 0000000000000000
>> (XEN) Xen call trace:
>> (XEN) [<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
>> (XEN) [<ffff82d0801834dc>] start_secondary+0x1bc/0x250
>> (XEN)
>> (XEN) Pagetable walk from 0000000000000001:
>> (XEN) L4[0x000] = 00000001bd8f0063 ffffffffffffffff
>> (XEN) L3[0x000] = 00000001bd8ef063 ffffffffffffffff
>> (XEN) L2[0x000] = 00000001bd8ee063 ffffffffffffffff
>> (XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
>> (XEN)
>> (XEN) ****************************************
>> (XEN) Panic on CPU 3:
>> (XEN) FATAL PAGE FAULT
>> (XEN) [error_code=0002]
>> (XEN) Faulting linear address: 0000000000000001
>> (XEN) ****************************************
>> (XEN)
>> (XEN) Reboot in five seconds...
>>
>> set_cpu_sibling_map+0x3f is the second cpumask_set_cpu() call in
>> set_cpu_sibling_map():
>> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/smpboot.c;h=0
>> 94699286f4f6962942024ec8b2b24c7b7996cc0;hb=78833c04250416f1870c458309d3ac0e5c
>> f915fd#l261
> I suppose cpu_to_socket(cpu) returns a value for which the
> socket_cpumask[] entry didn't get set up yet. But to prove that,
> we'd need to see the disassembly around the code location
> above, to be able to associate register values with variables.
>
> If that's the case, then I'd further guess that the CPUID
> information provided by Fusion isn't exactly as one would expect
> on real hardware. Whether we need to fix something, or can
> work around a quirk of theirs depends on the exact nature of
> the issue. Instrumenting code populating socket_cpumask[]
> would be a good first step.
Might also be interesting to see the logs with "apic_verbosity=debug",
and a debug hypervisor.
~Andrew
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-20 10:25 ` Jan Beulich
2015-11-20 10:37 ` Andrew Cooper
@ 2015-11-20 18:53 ` Ed Swierk
2015-11-21 1:21 ` Ed Swierk
1 sibling, 1 reply; 22+ messages in thread
From: Ed Swierk @ 2015-11-20 18:53 UTC (permalink / raw)
To: Jan Beulich; +Cc: Andrew Cooper, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 58251 bytes --]
I instrumented set_nr_sockets() and smp_store_cpu_info(), and re-ran with
varying numbers of CPUs.
With 4 CPUs, nr_sockets=4, so smp_store_cpu_info() exceeds the bounds of
the socket_cpumask array when socket=4 or 6.
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov 20
10:07:47 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.620 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) smp_store_cpu_info id=0
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) set_nr_sockets nr_sockets=4 num_processors=4 disabled_cpus=0 cpus=1
(XEN) Getting VERSION: 50015
(XEN) Getting VERSION: 50015
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) init IO_APIC IRQs
(XEN) IO-APIC (apicid-pin) 1-0, 1-16, 1-17, 1-18, 1-19, 1-20, 1-21, 1-22,
1-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #1 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #1......
(XEN) .... register #00: 01000000
(XEN) ....... : physical APIC id: 01
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170011
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0011
(XEN) .... register #02: 01000000
(XEN) ....... : arbitration: 01
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 001 01 0 0 0 0 0 1 1 28
(XEN) 02 001 01 0 0 0 0 0 1 1 F0
(XEN) 03 001 01 0 0 0 0 0 1 1 30
(XEN) 04 001 01 0 0 0 0 0 1 1 F1
(XEN) 05 001 01 0 0 0 0 0 1 1 38
(XEN) 06 001 01 0 0 0 0 0 1 1 40
(XEN) 07 001 01 0 0 0 0 0 1 1 48
(XEN) 08 001 01 0 0 0 0 0 1 1 50
(XEN) 09 001 01 0 0 0 0 0 1 1 58
(XEN) 0a 001 01 0 0 0 0 0 1 1 60
(XEN) 0b 001 01 0 0 0 0 0 1 1 68
(XEN) 0c 001 01 0 0 0 0 0 1 1 70
(XEN) 0d 001 01 0 0 0 0 0 1 1 78
(XEN) 0e 001 01 0 0 0 0 0 1 1 88
(XEN) 0f 001 01 0 0 0 0 0 1 1 90
(XEN) 10 000 00 1 0 0 0 0 0 0 00
(XEN) 11 000 00 1 0 0 0 0 0 0 00
(XEN) 12 000 00 1 0 0 0 0 0 0 00
(XEN) 13 000 00 1 0 0 0 0 0 0 00
(XEN) 14 000 00 1 0 0 0 0 0 0 00
(XEN) 15 000 00 1 0 0 0 0 0 0 00
(XEN) 16 000 00 1 0 0 0 0 0 0 00
(XEN) 17 000 00 1 0 0 0 0 0 0 00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ40 -> 0:1
(XEN) IRQ48 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ56 -> 0:5
(XEN) IRQ64 -> 0:6
(XEN) IRQ72 -> 0:7
(XEN) IRQ80 -> 0:8
(XEN) IRQ88 -> 0:9
(XEN) IRQ96 -> 0:10
(XEN) IRQ104 -> 0:11
(XEN) IRQ112 -> 0:12
(XEN) IRQ120 -> 0:13
(XEN) IRQ136 -> 0:14
(XEN) IRQ144 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 2592.9295 MHz.
(XEN) ..... host bus clock speed is 92.5953 MHz.
(XEN) ..... bus_scale = 0x5ed2
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) masked ExtINT on CPU#1
(XEN) smp_store_cpu_info id=1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=1 socket=2
socket_cpumask[socket]=0000000000000000
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) masked ExtINT on CPU#2
(XEN) smp_store_cpu_info id=2
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=2 socket=4
socket_cpumask[socket]=ffff8301bd8b5430
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 6 -> Node 0
(XEN) Booting processor 3/6 eip 8f000
(XEN) Initializing CPU#3
(XEN) masked ExtINT on CPU#3
(XEN) smp_store_cpu_info id=3
(XEN) CPU: Physical Processor ID: 6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=3 socket=6
socket_cpumask[socket]=0000000000000001
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) ----[ Xen-4.6.1-pre x86_64 debug=y Not tainted ]----
(XEN) CPU: 3
(XEN) RIP: e008:[<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
(XEN) RFLAGS: 0000000000010097 CONTEXT: hypervisor
(XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d58dc80
(XEN) rdx: 0000000000000006 rsi: 000000000000001e rdi: 0000000000000003
(XEN) rbp: ffff8301bd87fee0 rsp: ffff8301bd87fea0 r8: ffff8301bd87fea0
(XEN) r9: 0000000000000000 r10: 0000000000000001 r11: 0000000000000001
(XEN) r12: 00000000000000c8 r13: 0000000000000003 r14: 0000000000000000
(XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000001526a0
(XEN) cr3: 00000000bfc9f000 cr2: 0000000000000001
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff8301bd87fea0:
(XEN) 0000000300000000 0000000000000003 ffff8301bd87fee0 0000000000000000
(XEN) 00000000000000c8 0000000000000003 0000000000000000 0000000000000000
(XEN) ffff8301bd87ff10 ffff82d080188fc3 0000000000000001 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000003 ffff8300bfffb000 000000313d58dc80 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
(XEN) [<ffff82d080188fc3>] start_secondary+0x1fb/0x256
(XEN)
(XEN) Pagetable walk from 0000000000000001:
(XEN) L4[0x000] = 00000001bd8ec063 ffffffffffffffff
(XEN) L3[0x000] = 00000001bd8eb063 ffffffffffffffff
(XEN) L2[0x000] = 00000001bd8ea063 ffffffffffffffff
(XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 3:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000001
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
Similar issue with 8 CPUs.
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov 20
10:07:47 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF00E7, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF0405, EA6E (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF03DD, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF032B, 00B2 (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF02EF, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF019F, 0150 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF0177, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 08 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0a -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0c -> Node 0
(XEN) SRAT: PXM 0 -> APIC 0e -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled)
(XEN) Processor #8 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] enabled)
(XEN) Processor #10 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] enabled)
(XEN) Processor #12 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] enabled)
(XEN) Processor #14 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.575 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) smp_store_cpu_info id=0
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) set_nr_sockets nr_sockets=8 num_processors=8 disabled_cpus=0 cpus=1
(XEN) Getting VERSION: 50015
(XEN) Getting VERSION: 50015
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) init IO_APIC IRQs
(XEN) IO-APIC (apicid-pin) 1-0, 1-16, 1-17, 1-18, 1-19, 1-20, 1-21, 1-22,
1-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #1 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #1......
(XEN) .... register #00: 01000000
(XEN) ....... : physical APIC id: 01
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170011
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0011
(XEN) .... register #02: 01000000
(XEN) ....... : arbitration: 01
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 001 01 0 0 0 0 0 1 1 28
(XEN) 02 001 01 0 0 0 0 0 1 1 F0
(XEN) 03 001 01 0 0 0 0 0 1 1 30
(XEN) 04 001 01 0 0 0 0 0 1 1 F1
(XEN) 05 001 01 0 0 0 0 0 1 1 38
(XEN) 06 001 01 0 0 0 0 0 1 1 40
(XEN) 07 001 01 0 0 0 0 0 1 1 48
(XEN) 08 001 01 0 0 0 0 0 1 1 50
(XEN) 09 001 01 0 0 0 0 0 1 1 58
(XEN) 0a 001 01 0 0 0 0 0 1 1 60
(XEN) 0b 001 01 0 0 0 0 0 1 1 68
(XEN) 0c 001 01 0 0 0 0 0 1 1 70
(XEN) 0d 001 01 0 0 0 0 0 1 1 78
(XEN) 0e 001 01 0 0 0 0 0 1 1 88
(XEN) 0f 001 01 0 0 0 0 0 1 1 90
(XEN) 10 000 00 1 0 0 0 0 0 0 00
(XEN) 11 000 00 1 0 0 0 0 0 0 00
(XEN) 12 000 00 1 0 0 0 0 0 0 00
(XEN) 13 000 00 1 0 0 0 0 0 0 00
(XEN) 14 000 00 1 0 0 0 0 0 0 00
(XEN) 15 000 00 1 0 0 0 0 0 0 00
(XEN) 16 000 00 1 0 0 0 0 0 0 00
(XEN) 17 000 00 1 0 0 0 0 0 0 00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ40 -> 0:1
(XEN) IRQ48 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ56 -> 0:5
(XEN) IRQ64 -> 0:6
(XEN) IRQ72 -> 0:7
(XEN) IRQ80 -> 0:8
(XEN) IRQ88 -> 0:9
(XEN) IRQ96 -> 0:10
(XEN) IRQ104 -> 0:11
(XEN) IRQ112 -> 0:12
(XEN) IRQ120 -> 0:13
(XEN) IRQ136 -> 0:14
(XEN) IRQ144 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 2592.9336 MHz.
(XEN) ..... host bus clock speed is 92.6030 MHz.
(XEN) ..... bus_scale = 0x5ed5
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 64 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) masked ExtINT on CPU#1
(XEN) smp_store_cpu_info id=1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=1 socket=2
socket_cpumask[socket]=0000000000000000
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) masked ExtINT on CPU#2
(XEN) smp_store_cpu_info id=2
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=2 socket=4
socket_cpumask[socket]=0000000000000000
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 6 -> Node 0
(XEN) Booting processor 3/6 eip 8f000
(XEN) Initializing CPU#3
(XEN) masked ExtINT on CPU#3
(XEN) smp_store_cpu_info id=3
(XEN) CPU: Physical Processor ID: 6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=3 socket=6
socket_cpumask[socket]=0000000000000000
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 4 APIC 8 -> Node 0
(XEN) Booting processor 4/8 eip 8f000
(XEN) Initializing CPU#4
(XEN) masked ExtINT on CPU#4
(XEN) smp_store_cpu_info id=4
(XEN) CPU: Physical Processor ID: 8
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU4: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=4 socket=8
socket_cpumask[socket]=ffff8301bd865430
(XEN) CPU4: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 5 APIC 10 -> Node 0
(XEN) Booting processor 5/10 eip 8f000
(XEN) Initializing CPU#5
(XEN) masked ExtINT on CPU#5
(XEN) smp_store_cpu_info id=5
(XEN) CPU: Physical Processor ID: 10
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU5: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=5 socket=10
socket_cpumask[socket]=0000000000000001
(XEN) CPU5: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) ----[ Xen-4.6.1-pre x86_64 debug=y Not tainted ]----
(XEN) CPU: 5
(XEN) RIP: e008:[<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
(XEN) RFLAGS: 0000000000010093 CONTEXT: hypervisor
(XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d571c80
(XEN) rdx: 000000000000000a rsi: 0000000000000026 rdi: 0000000000000005
(XEN) rbp: ffff8301bd8a7ee0 rsp: ffff8301bd8a7ea0 r8: ffff8301bd8a7ea0
(XEN) r9: 0000000000000000 r10: 0000000000000001 r11: 0000000000000001
(XEN) r12: 00000000000000c8 r13: 0000000000000005 r14: 0000000000000000
(XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000001526a0
(XEN) cr3: 00000000bfc9f000 cr2: 0000000000000001
(XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
(XEN) Xen stack trace from rsp=ffff8301bd8a7ea0:
(XEN) 0000000500000000 0000000000000005 ffff8301bd8a7ee0 0000000000000000
(XEN) 00000000000000c8 0000000000000005 0000000000000000 0000000000000000
(XEN) ffff8301bd8a7f10 ffff82d080188fc3 0000000000000001 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000000 0000000000000000 0000000000000000 0000000000000000
(XEN) 0000000000000005 ffff8300bfff9000 000000313d571c80 0000000000000000
(XEN) Xen call trace:
(XEN) [<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
(XEN) [<ffff82d080188fc3>] start_secondary+0x1fb/0x256
(XEN)
(XEN) Pagetable walk from 0000000000000001:
(XEN) L4[0x000] = 00000001bd8ec063 ffffffffffffffff
(XEN) L3[0x000] = 00000001bd8eb063 ffffffffffffffff
(XEN) L2[0x000] = 00000001bd8ea063 ffffffffffffffff
(XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
(XEN)
(XEN) ****************************************
(XEN) Panic on CPU 5:
(XEN) FATAL PAGE FAULT
(XEN) [error_code=0002]
(XEN) Faulting linear address: 0000000000000001
(XEN) ****************************************
(XEN)
(XEN) Reboot in five seconds...
(XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
With 3 CPUs the situation is completely different. nr_sockets=1, but socket
is always 0 in smp_store_cpu_info().
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov 20
10:07:47 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF0396, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF061E, E855 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF05F6, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF058A, 006C (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF054E, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF044E, 0100 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF0426, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 01 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x03] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 3, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 3 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 568 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.635 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) smp_store_cpu_info id=0
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) set_nr_sockets nr_sockets=1 num_processors=3 disabled_cpus=0 cpus=3
(XEN) Getting VERSION: 50015
(XEN) Getting VERSION: 50015
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) init IO_APIC IRQs
(XEN) IO-APIC (apicid-pin) 3-0, 3-16, 3-17, 3-18, 3-19, 3-20, 3-21, 3-22,
3-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #3 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #3......
(XEN) .... register #00: 03000000
(XEN) ....... : physical APIC id: 03
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170011
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0011
(XEN) .... register #02: 03000000
(XEN) ....... : arbitration: 03
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 001 01 0 0 0 0 0 1 1 28
(XEN) 02 001 01 0 0 0 0 0 1 1 F0
(XEN) 03 001 01 0 0 0 0 0 1 1 30
(XEN) 04 001 01 0 0 0 0 0 1 1 F1
(XEN) 05 001 01 0 0 0 0 0 1 1 38
(XEN) 06 001 01 0 0 0 0 0 1 1 40
(XEN) 07 001 01 0 0 0 0 0 1 1 48
(XEN) 08 001 01 0 0 0 0 0 1 1 50
(XEN) 09 001 01 0 0 0 0 0 1 1 58
(XEN) 0a 001 01 0 0 0 0 0 1 1 60
(XEN) 0b 001 01 0 0 0 0 0 1 1 68
(XEN) 0c 001 01 0 0 0 0 0 1 1 70
(XEN) 0d 001 01 0 0 0 0 0 1 1 78
(XEN) 0e 001 01 0 0 0 0 0 1 1 88
(XEN) 0f 001 01 0 0 0 0 0 1 1 90
(XEN) 10 000 00 1 0 0 0 0 0 0 00
(XEN) 11 000 00 1 0 0 0 0 0 0 00
(XEN) 12 000 00 1 0 0 0 0 0 0 00
(XEN) 13 000 00 1 0 0 0 0 0 0 00
(XEN) 14 000 00 1 0 0 0 0 0 0 00
(XEN) 15 000 00 1 0 0 0 0 0 0 00
(XEN) 16 000 00 1 0 0 0 0 0 0 00
(XEN) 17 000 00 1 0 0 0 0 0 0 00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ40 -> 0:1
(XEN) IRQ48 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ56 -> 0:5
(XEN) IRQ64 -> 0:6
(XEN) IRQ72 -> 0:7
(XEN) IRQ80 -> 0:8
(XEN) IRQ88 -> 0:9
(XEN) IRQ96 -> 0:10
(XEN) IRQ104 -> 0:11
(XEN) IRQ112 -> 0:12
(XEN) IRQ120 -> 0:13
(XEN) IRQ136 -> 0:14
(XEN) IRQ144 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 2592.9209 MHz.
(XEN) ..... host bus clock speed is 92.5952 MHz.
(XEN) ..... bus_scale = 0x5ed2
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 1 -> Node 0
(XEN) Booting processor 1/1 eip 8f000
(XEN) Initializing CPU#1
(XEN) masked ExtINT on CPU#1
(XEN) smp_store_cpu_info id=1
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=1 socket=0
socket_cpumask[socket]=ffff8301bd8e5460
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 2 -> Node 0
(XEN) Booting processor 2/2 eip 8f000
(XEN) Initializing CPU#2
(XEN) masked ExtINT on CPU#2
(XEN) smp_store_cpu_info id=2
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=2 socket=0
socket_cpumask[socket]=ffff8301bd8e5460
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) Brought up 3 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) Dom0 has maximum 408 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x68f6d8
(XEN) elf_parse_binary: phdr: paddr=0x1690000 memsz=0x409000
(XEN) elf_parse_binary: phdr: paddr=0x1a99000 memsz=0x32c120
(XEN) elf_parse_binary: phdr: paddr=0x1dc6000 memsz=0x1000
(XEN) elf_parse_binary: phdr: paddr=0x1dc7000 memsz=0x1d13c0
(XEN) elf_parse_binary: phdr: paddr=0x1f99000 memsz=0x53000
(XEN) elf_parse_binary: phdr: paddr=0x1fec000 memsz=0x1250
(XEN) elf_parse_binary: phdr: paddr=0x1fee000 memsz=0x1012000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x3000000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81f991f0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES =
"!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: SUPPORTED_FEATURES = 0x801
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN) virt_base = 0xffffffff80000000
(XEN) elf_paddr_offset = 0x0
(XEN) virt_offset = 0xffffffff80000000
(XEN) virt_kstart = 0xffffffff81000000
(XEN) virt_kend = 0xffffffff83000000
(XEN) virt_entry = 0xffffffff81f991f0
(XEN) p2m_base = 0xffffffffffffffff
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
be allocated)
(XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
(XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
(XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
(XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
(XEN) Page tables: ffffffff8412d000->ffffffff84152000
(XEN) Boot stack: ffffffff84152000->ffffffff84153000
(XEN) TOTAL: ffffffff80000000->ffffffff84400000
(XEN) ENTRY ADDRESS: ffffffff81f991f0
(XEN) Dom0 has maximum 3 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8168f6d8
(XEN) elf_load_binary: phdr 1 at 0xffffffff81690000 -> 0xffffffff81a99000
(XEN) elf_load_binary: phdr 2 at 0xffffffff81a99000 -> 0xffffffff81dc5120
(XEN) elf_load_binary: phdr 3 at 0xffffffff81dc6000 -> 0xffffffff81dc7000
(XEN) elf_load_binary: phdr 4 at 0xffffffff81dc7000 -> 0xffffffff81f983c0
(XEN) elf_load_binary: phdr 5 at 0xffffffff81f99000 -> 0xffffffff81fec000
(XEN) elf_load_binary: phdr 6 at 0xffffffff81fec000 -> 0xffffffff81fed250
(XEN) elf_load_binary: phdr 7 at 0xffffffff81fee000 -> 0xffffffff820ce000
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 308kB init memory.
mapping kernel into physical memory
about to get started...
Similarly with 6 CPUs, nr_sockets=2, and socket is always 0 or 1 in
smp_store_cpu_info().
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov 20
10:07:47 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF01F9, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF04DB, E998 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF04B3, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF041D, 0096 (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF03E1, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF02B1, 0130 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF0289, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 01 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 05 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
(XEN) Processor #1 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x05] enabled)
(XEN) Processor #5 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x03] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 3, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 6 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 1144 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.566 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) smp_store_cpu_info id=0
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) set_nr_sockets nr_sockets=2 num_processors=6 disabled_cpus=0 cpus=3
(XEN) Getting VERSION: 50015
(XEN) Getting VERSION: 50015
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) init IO_APIC IRQs
(XEN) IO-APIC (apicid-pin) 3-0, 3-16, 3-17, 3-18, 3-19, 3-20, 3-21, 3-22,
3-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #3 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #3......
(XEN) .... register #00: 03000000
(XEN) ....... : physical APIC id: 03
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170011
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0011
(XEN) .... register #02: 03000000
(XEN) ....... : arbitration: 03
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 001 01 0 0 0 0 0 1 1 28
(XEN) 02 001 01 0 0 0 0 0 1 1 F0
(XEN) 03 001 01 0 0 0 0 0 1 1 30
(XEN) 04 001 01 0 0 0 0 0 1 1 F1
(XEN) 05 001 01 0 0 0 0 0 1 1 38
(XEN) 06 001 01 0 0 0 0 0 1 1 40
(XEN) 07 001 01 0 0 0 0 0 1 1 48
(XEN) 08 001 01 0 0 0 0 0 1 1 50
(XEN) 09 001 01 0 0 0 0 0 1 1 58
(XEN) 0a 001 01 0 0 0 0 0 1 1 60
(XEN) 0b 001 01 0 0 0 0 0 1 1 68
(XEN) 0c 001 01 0 0 0 0 0 1 1 70
(XEN) 0d 001 01 0 0 0 0 0 1 1 78
(XEN) 0e 001 01 0 0 0 0 0 1 1 88
(XEN) 0f 001 01 0 0 0 0 0 1 1 90
(XEN) 10 000 00 1 0 0 0 0 0 0 00
(XEN) 11 000 00 1 0 0 0 0 0 0 00
(XEN) 12 000 00 1 0 0 0 0 0 0 00
(XEN) 13 000 00 1 0 0 0 0 0 0 00
(XEN) 14 000 00 1 0 0 0 0 0 0 00
(XEN) 15 000 00 1 0 0 0 0 0 0 00
(XEN) 16 000 00 1 0 0 0 0 0 0 00
(XEN) 17 000 00 1 0 0 0 0 0 0 00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ40 -> 0:1
(XEN) IRQ48 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ56 -> 0:5
(XEN) IRQ64 -> 0:6
(XEN) IRQ72 -> 0:7
(XEN) IRQ80 -> 0:8
(XEN) IRQ88 -> 0:9
(XEN) IRQ96 -> 0:10
(XEN) IRQ104 -> 0:11
(XEN) IRQ112 -> 0:12
(XEN) IRQ120 -> 0:13
(XEN) IRQ136 -> 0:14
(XEN) IRQ144 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 2592.9460 MHz.
(XEN) ..... host bus clock speed is 92.5859 MHz.
(XEN) ..... bus_scale = 0x5ed0
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 64 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 1 -> Node 0
(XEN) Booting processor 1/1 eip 8f000
(XEN) Initializing CPU#1
(XEN) masked ExtINT on CPU#1
(XEN) smp_store_cpu_info id=1
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=1 socket=0
socket_cpumask[socket]=ffff8301bd84d460
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 2 -> Node 0
(XEN) Booting processor 2/2 eip 8f000
(XEN) Initializing CPU#2
(XEN) masked ExtINT on CPU#2
(XEN) smp_store_cpu_info id=2
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: Processor Core ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=2 socket=0
socket_cpumask[socket]=ffff8301bd84d460
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 4 -> Node 0
(XEN) Booting processor 3/4 eip 8f000
(XEN) Initializing CPU#3
(XEN) masked ExtINT on CPU#3
(XEN) smp_store_cpu_info id=3
(XEN) CPU: Physical Processor ID: 1
(XEN) CPU: Processor Core ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=3 socket=1
socket_cpumask[socket]=0000000000000000
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 4 APIC 5 -> Node 0
(XEN) Booting processor 4/5 eip 8f000
(XEN) Initializing CPU#4
(XEN) masked ExtINT on CPU#4
(XEN) smp_store_cpu_info id=4
(XEN) CPU: Physical Processor ID: 1
(XEN) CPU: Processor Core ID: 1
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU4: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=4 socket=1
socket_cpumask[socket]=ffff8301bd84df90
(XEN) CPU4: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 5 APIC 6 -> Node 0
(XEN) Booting processor 5/6 eip 8f000
(XEN) Initializing CPU#5
(XEN) masked ExtINT on CPU#5
(XEN) smp_store_cpu_info id=5
(XEN) CPU: Physical Processor ID: 1
(XEN) CPU: Processor Core ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU5: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=5 socket=1
socket_cpumask[socket]=ffff8301bd84df90
(XEN) CPU5: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) Brought up 6 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) Dom0 has maximum 600 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x68f6d8
(XEN) elf_parse_binary: phdr: paddr=0x1690000 memsz=0x409000
(XEN) elf_parse_binary: phdr: paddr=0x1a99000 memsz=0x32c120
(XEN) elf_parse_binary: phdr: paddr=0x1dc6000 memsz=0x1000
(XEN) elf_parse_binary: phdr: paddr=0x1dc7000 memsz=0x1d13c0
(XEN) elf_parse_binary: phdr: paddr=0x1f99000 memsz=0x53000
(XEN) elf_parse_binary: phdr: paddr=0x1fec000 memsz=0x1250
(XEN) elf_parse_binary: phdr: paddr=0x1fee000 memsz=0x1012000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x3000000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81f991f0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES =
"!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: SUPPORTED_FEATURES = 0x801
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN) virt_base = 0xffffffff80000000
(XEN) elf_paddr_offset = 0x0
(XEN) virt_offset = 0xffffffff80000000
(XEN) virt_kstart = 0xffffffff81000000
(XEN) virt_kend = 0xffffffff83000000
(XEN) virt_entry = 0xffffffff81f991f0
(XEN) p2m_base = 0xffffffffffffffff
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
be allocated)
(XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
(XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
(XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
(XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
(XEN) Page tables: ffffffff8412d000->ffffffff84152000
(XEN) Boot stack: ffffffff84152000->ffffffff84153000
(XEN) TOTAL: ffffffff80000000->ffffffff84400000
(XEN) ENTRY ADDRESS: ffffffff81f991f0
(XEN) Dom0 has maximum 6 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8168f6d8
(XEN) elf_load_binary: phdr 1 at 0xffffffff81690000 -> 0xffffffff81a99000
(XEN) elf_load_binary: phdr 2 at 0xffffffff81a99000 -> 0xffffffff81dc5120
(XEN) elf_load_binary: phdr 3 at 0xffffffff81dc6000 -> 0xffffffff81dc7000
(XEN) elf_load_binary: phdr 4 at 0xffffffff81dc7000 -> 0xffffffff81f983c0
(XEN) elf_load_binary: phdr 5 at 0xffffffff81f99000 -> 0xffffffff81fec000
(XEN) elf_load_binary: phdr 6 at 0xffffffff81fec000 -> 0xffffffff81fed250
(XEN) elf_load_binary: phdr 7 at 0xffffffff81fee000 -> 0xffffffff820ce000
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 308kB init memory.
mapping kernel into physical memory
about to get started...
So it looks like for power-of-2 CPUs, Fusion is providing a separate socket
per CPU, numbered 0, 2, 4, ...
For multiple-of-3 CPUs, Fusion is providing a socket for every 3 CPUs,
numbered 0, 1, ...
--Ed
On Fri, Nov 20, 2015 at 2:25 AM, Jan Beulich <JBeulich@suse.com> wrote:
> >>> On 20.11.15 at 02:22, <eswierk@skyportsystems.com> wrote:
> > (XEN) ----[ Xen-4.6.1-pre x86_64 debug=n Not tainted ]----
> > (XEN) CPU: 3
> > (XEN) RIP: e008:[<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
> > (XEN) RFLAGS: 0000000000010006 CONTEXT: hypervisor
> > (XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx:
> 000000313d5b4080
> > (XEN) rdx: 0000000000000006 rsi: 0000000000000000 rdi:
> 0000000000000003
> > (XEN) rbp: 0000000000000300 rsp: ffff8301bd87fe90 r8:
> ffff8301bd878000
> > (XEN) r9: 000000313d5b4080 r10: 0000000000000001 r11:
> 0000000000000001
> > (XEN) r12: ffff82d0802fd500 r13: 0000000000000000 r14:
> 0000000000000000
> > (XEN) r15: 0000000000000003 cr0: 000000008005003b cr4:
> 00000000001526a0
> > (XEN) cr3: 00000000bfc75000 cr2: 0000000000000001
> > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> > (XEN) Xen stack trace from rsp=ffff8301bd87fe90:
> > (XEN) 00000003802fd800 0000000000000018 0000000000000000
> 0000010000000000
> > (XEN) ffff82d0802fd800 0000000000000000 00000000000000c8
> 0000000000000003
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> ffff82d0801834dc
> > (XEN) 0000000000000000 0000000000000001 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> > (XEN) 0000000000000000 0000000000000000 0000000000000003
> ffff8300bfafc000
> > (XEN) 000000313d5b4080 0000000000000000
> > (XEN) Xen call trace:
> > (XEN) [<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
> > (XEN) [<ffff82d0801834dc>] start_secondary+0x1bc/0x250
> > (XEN)
> > (XEN) Pagetable walk from 0000000000000001:
> > (XEN) L4[0x000] = 00000001bd8f0063 ffffffffffffffff
> > (XEN) L3[0x000] = 00000001bd8ef063 ffffffffffffffff
> > (XEN) L2[0x000] = 00000001bd8ee063 ffffffffffffffff
> > (XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
> > (XEN)
> > (XEN) ****************************************
> > (XEN) Panic on CPU 3:
> > (XEN) FATAL PAGE FAULT
> > (XEN) [error_code=0002]
> > (XEN) Faulting linear address: 0000000000000001
> > (XEN) ****************************************
> > (XEN)
> > (XEN) Reboot in five seconds...
> >
> > set_cpu_sibling_map+0x3f is the second cpumask_set_cpu() call in
> > set_cpu_sibling_map():
> >
> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/smpboot.c;h=0
> >
> 94699286f4f6962942024ec8b2b24c7b7996cc0;hb=78833c04250416f1870c458309d3ac0e5c
> > f915fd#l261
>
> I suppose cpu_to_socket(cpu) returns a value for which the
> socket_cpumask[] entry didn't get set up yet. But to prove that,
> we'd need to see the disassembly around the code location
> above, to be able to associate register values with variables.
>
> If that's the case, then I'd further guess that the CPUID
> information provided by Fusion isn't exactly as one would expect
> on real hardware. Whether we need to fix something, or can
> work around a quirk of theirs depends on the exact nature of
> the issue. Instrumenting code populating socket_cpumask[]
> would be a good first step.
>
> Jan
>
>
[-- Attachment #1.2: Type: text/html, Size: 74425 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-20 18:53 ` Ed Swierk
@ 2015-11-21 1:21 ` Ed Swierk
2015-11-23 1:10 ` Chao Peng
2015-11-23 10:18 ` Jan Beulich
0 siblings, 2 replies; 22+ messages in thread
From: Ed Swierk @ 2015-11-21 1:21 UTC (permalink / raw)
To: Jan Beulich; +Cc: Andrew Cooper, Chao Peng, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 61830 bytes --]
The problem is that the index of the socket_cpumask array is derived via
cpu_to_socket() from the APIC ID of the processor in a given socket, but
the size of the array is computed based on nr_sockets, which is not
necessarily equal to the maximum APIC ID.
Sizing the socket_cpumask to MAX_APICS rather than nr_sockets seems safer,
though a bit wasteful. I verified that this change fixes the boot crash
with 4 or 8 CPUs on VMware Fusion.
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -819,7 +819,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
set_nr_sockets();
- socket_cpumask = xzalloc_array(cpumask_t *, nr_sockets);
+ socket_cpumask = xzalloc_array(cpumask_t *, MAX_APICS);
if ( socket_cpumask == NULL ||
(socket_cpumask[cpu_to_socket(0)] = xzalloc(cpumask_t)) == NULL )
panic("No memory for socket CPU siblings map");
On Fri, Nov 20, 2015 at 10:53 AM, Ed Swierk <eswierk@skyportsystems.com>
wrote:
> I instrumented set_nr_sockets() and smp_store_cpu_info(), and re-ran with
> varying numbers of CPUs.
>
> With 4 CPUs, nr_sockets=4, so smp_store_cpu_info() exceeds the bounds of
> the socket_cpumask array when socket=4 or 6.
>
> Loading xen-4.6-amd64.gz... ok
> Loading vmlinuz-3.14.51-grsec-dock... ok
> Loading initrd.img-3.14.51-grsec-dock... ok
> (XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
> eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov
> 20 10:07:47 PST 2015
> (XEN) Bootloader: SYSLINUX 4.05 20140113
> (XEN) Command line: console=com1,vga com1=115200 no-bootscrub
> dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f800 (usable)
> (XEN) 000000000009f800 - 00000000000a0000 (reserved)
> (XEN) 00000000000dc000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bfef0000 (usable)
> (XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
> (XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
> (XEN) 00000000bff00000 - 00000000c0000000 (usable)
> (XEN) 00000000f0000000 - 00000000f8000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee01000 (reserved)
> (XEN) 00000000fffe0000 - 0000000100000000 (reserved)
> (XEN) 0000000100000000 - 00000001c0000000 (usable)
> (XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
> (XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
> (XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
> (XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
> (XEN) ACPI: FACS BFEFFFC0, 0040
> (XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
> (XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
> (XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
> (XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
> (XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
> (XEN) System RAM: 6143MB (6291004kB)
> (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
> (XEN) SRAT: Node 0 PXM 0 0-a0000
> (XEN) SRAT: Node 0 PXM 0 100000-10000000
> (XEN) SRAT: Node 0 PXM 0 10000000-c0000000
> (XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
> (XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
> (XEN) NUMA: Using 8 for the hash shift.
> (XEN) Domain heap initialised
> (XEN) found SMP MP-table at 000f6a80
> (XEN) DMI present.
> (XEN) APIC boot state is 'xapic'
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
> (XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> (XEN) Processor #2 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> (XEN) Processor #4 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> (XEN) Processor #6 6:6 APIC version 21
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ERST table was not found
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
> (XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
> (XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
> (XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
> (XEN) Not enabling x2APIC: depends on iommu_supports_eim.
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
> (XEN) CPU0: No MCE banks present. Machine check support disabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2592.620 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
> (XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
> (XEN) PCI: MCFG area at f0000000 reserved in E820
> (XEN) PCI: Using MCFG for segment 0000 bus 00-7f
> (XEN) I/O virtualisation disabled
> (XEN) smp_store_cpu_info id=0
> (XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) set_nr_sockets nr_sockets=4 num_processors=4 disabled_cpus=0 cpus=1
> (XEN) Getting VERSION: 50015
> (XEN) Getting VERSION: 50015
> (XEN) Getting ID: 0
> (XEN) Getting LVT0: 700
> (XEN) Getting LVT1: 400
> (XEN) enabled ExtINT on CPU#0
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) init IO_APIC IRQs
> (XEN) IO-APIC (apicid-pin) 1-0, 1-16, 1-17, 1-18, 1-19, 1-20, 1-21, 1-22,
> 1-23 not connected.
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) number of MP IRQ sources: 15.
> (XEN) number of IO-APIC #1 registers: 24.
> (XEN) testing the IO APIC.......................
> (XEN) IO APIC #1......
> (XEN) .... register #00: 01000000
> (XEN) ....... : physical APIC id: 01
> (XEN) ....... : Delivery Type: 0
> (XEN) ....... : LTS : 0
> (XEN) .... register #01: 00170011
> (XEN) ....... : max redirection entries: 0017
> (XEN) ....... : PRQ implemented: 0
> (XEN) ....... : IO APIC version: 0011
> (XEN) .... register #02: 01000000
> (XEN) ....... : arbitration: 01
> (XEN) .... IRQ redirection table:
> (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
> (XEN) 00 000 00 1 0 0 0 0 0 0 00
> (XEN) 01 001 01 0 0 0 0 0 1 1 28
> (XEN) 02 001 01 0 0 0 0 0 1 1 F0
> (XEN) 03 001 01 0 0 0 0 0 1 1 30
> (XEN) 04 001 01 0 0 0 0 0 1 1 F1
> (XEN) 05 001 01 0 0 0 0 0 1 1 38
> (XEN) 06 001 01 0 0 0 0 0 1 1 40
> (XEN) 07 001 01 0 0 0 0 0 1 1 48
> (XEN) 08 001 01 0 0 0 0 0 1 1 50
> (XEN) 09 001 01 0 0 0 0 0 1 1 58
> (XEN) 0a 001 01 0 0 0 0 0 1 1 60
> (XEN) 0b 001 01 0 0 0 0 0 1 1 68
> (XEN) 0c 001 01 0 0 0 0 0 1 1 70
> (XEN) 0d 001 01 0 0 0 0 0 1 1 78
> (XEN) 0e 001 01 0 0 0 0 0 1 1 88
> (XEN) 0f 001 01 0 0 0 0 0 1 1 90
> (XEN) 10 000 00 1 0 0 0 0 0 0 00
> (XEN) 11 000 00 1 0 0 0 0 0 0 00
> (XEN) 12 000 00 1 0 0 0 0 0 0 00
> (XEN) 13 000 00 1 0 0 0 0 0 0 00
> (XEN) 14 000 00 1 0 0 0 0 0 0 00
> (XEN) 15 000 00 1 0 0 0 0 0 0 00
> (XEN) 16 000 00 1 0 0 0 0 0 0 00
> (XEN) 17 000 00 1 0 0 0 0 0 0 00
> (XEN) Using vector-based indexing
> (XEN) IRQ to pin mappings:
> (XEN) IRQ240 -> 0:2
> (XEN) IRQ40 -> 0:1
> (XEN) IRQ48 -> 0:3
> (XEN) IRQ241 -> 0:4
> (XEN) IRQ56 -> 0:5
> (XEN) IRQ64 -> 0:6
> (XEN) IRQ72 -> 0:7
> (XEN) IRQ80 -> 0:8
> (XEN) IRQ88 -> 0:9
> (XEN) IRQ96 -> 0:10
> (XEN) IRQ104 -> 0:11
> (XEN) IRQ112 -> 0:12
> (XEN) IRQ120 -> 0:13
> (XEN) IRQ136 -> 0:14
> (XEN) IRQ144 -> 0:15
> (XEN) .................................... done.
> (XEN) Using local APIC timer interrupts.
> (XEN) calibrating APIC timer ...
> (XEN) ..... CPU clock speed is 2592.9295 MHz.
> (XEN) ..... host bus clock speed is 92.5953 MHz.
> (XEN) ..... bus_scale = 0x5ed2
> (XEN) Platform timer is 3.579MHz ACPI PM Timer
> (XEN) Allocated console ring of 32 KiB.
> (XEN) mwait-idle: MWAIT substates: 0x10
> (XEN) mwait-idle: v0.4 model 0x46
> (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC TPR shadow
> (XEN) - Extended Page Tables (EPT)
> (XEN) - Virtual-Processor Identifiers (VPID)
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) not detected
> (XEN) HVM: PVH mode not supported on this platform
> (XEN) CPU 0 APIC 0 -> Node 0
> (XEN) CPU 1 APIC 2 -> Node 0
> (XEN) Booting processor 1/2 eip 8f000
> (XEN) Initializing CPU#1
> (XEN) masked ExtINT on CPU#1
> (XEN) smp_store_cpu_info id=1
> (XEN) CPU: Physical Processor ID: 2
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU1: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=1 socket=2
> socket_cpumask[socket]=0000000000000000
> (XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 2 APIC 4 -> Node 0
> (XEN) Booting processor 2/4 eip 8f000
> (XEN) Initializing CPU#2
> (XEN) masked ExtINT on CPU#2
> (XEN) smp_store_cpu_info id=2
> (XEN) CPU: Physical Processor ID: 4
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU2: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=2 socket=4
> socket_cpumask[socket]=ffff8301bd8b5430
> (XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 3 APIC 6 -> Node 0
> (XEN) Booting processor 3/6 eip 8f000
> (XEN) Initializing CPU#3
> (XEN) masked ExtINT on CPU#3
> (XEN) smp_store_cpu_info id=3
> (XEN) CPU: Physical Processor ID: 6
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU3: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=3 socket=6
> socket_cpumask[socket]=0000000000000001
> (XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) ----[ Xen-4.6.1-pre x86_64 debug=y Not tainted ]----
> (XEN) CPU: 3
> (XEN) RIP: e008:[<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
> (XEN) RFLAGS: 0000000000010097 CONTEXT: hypervisor
> (XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d58dc80
> (XEN) rdx: 0000000000000006 rsi: 000000000000001e rdi: 0000000000000003
> (XEN) rbp: ffff8301bd87fee0 rsp: ffff8301bd87fea0 r8: ffff8301bd87fea0
> (XEN) r9: 0000000000000000 r10: 0000000000000001 r11: 0000000000000001
> (XEN) r12: 00000000000000c8 r13: 0000000000000003 r14: 0000000000000000
> (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000001526a0
> (XEN) cr3: 00000000bfc9f000 cr2: 0000000000000001
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff8301bd87fea0:
> (XEN) 0000000300000000 0000000000000003 ffff8301bd87fee0
> 0000000000000000
> (XEN) 00000000000000c8 0000000000000003 0000000000000000
> 0000000000000000
> (XEN) ffff8301bd87ff10 ffff82d080188fc3 0000000000000001
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000003 ffff8300bfffb000 000000313d58dc80
> 0000000000000000
> (XEN) Xen call trace:
> (XEN) [<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
> (XEN) [<ffff82d080188fc3>] start_secondary+0x1fb/0x256
> (XEN)
> (XEN) Pagetable walk from 0000000000000001:
> (XEN) L4[0x000] = 00000001bd8ec063 ffffffffffffffff
> (XEN) L3[0x000] = 00000001bd8eb063 ffffffffffffffff
> (XEN) L2[0x000] = 00000001bd8ea063 ffffffffffffffff
> (XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 3:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0002]
> (XEN) Faulting linear address: 0000000000000001
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
>
> Similar issue with 8 CPUs.
>
> Loading xen-4.6-amd64.gz... ok
> Loading vmlinuz-3.14.51-grsec-dock... ok
> Loading initrd.img-3.14.51-grsec-dock... ok
> (XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
> eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov
> 20 10:07:47 PST 2015
> (XEN) Bootloader: SYSLINUX 4.05 20140113
> (XEN) Command line: console=com1,vga com1=115200 no-bootscrub
> dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f800 (usable)
> (XEN) 000000000009f800 - 00000000000a0000 (reserved)
> (XEN) 00000000000dc000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bfef0000 (usable)
> (XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
> (XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
> (XEN) 00000000bff00000 - 00000000c0000000 (usable)
> (XEN) 00000000f0000000 - 00000000f8000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee01000 (reserved)
> (XEN) 00000000fffe0000 - 0000000100000000 (reserved)
> (XEN) 0000000100000000 - 00000001c0000000 (usable)
> (XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
> (XEN) ACPI: XSDT BFEF00E7, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
> (XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
> (XEN) ACPI: DSDT BFEF0405, EA6E (r1 PTLTD Custom 6040000 MSFT 3000001)
> (XEN) ACPI: FACS BFEFFFC0, 0040
> (XEN) ACPI: BOOT BFEF03DD, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
> (XEN) ACPI: APIC BFEF032B, 00B2 (r1 PTLTD APIC 6040000 LTP 0)
> (XEN) ACPI: MCFG BFEF02EF, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
> (XEN) ACPI: SRAT BFEF019F, 0150 (r2 VMWARE MEMPLUG 6040000 VMW 1)
> (XEN) ACPI: WAET BFEF0177, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
> (XEN) System RAM: 6143MB (6291004kB)
> (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 08 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 0a -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 0c -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 0e -> Node 0
> (XEN) SRAT: Node 0 PXM 0 0-a0000
> (XEN) SRAT: Node 0 PXM 0 100000-10000000
> (XEN) SRAT: Node 0 PXM 0 10000000-c0000000
> (XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
> (XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
> (XEN) NUMA: Using 8 for the hash shift.
> (XEN) Domain heap initialised
> (XEN) found SMP MP-table at 000f6a80
> (XEN) DMI present.
> (XEN) APIC boot state is 'xapic'
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
> (XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> (XEN) Processor #2 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> (XEN) Processor #4 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> (XEN) Processor #6 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x08] enabled)
> (XEN) Processor #8 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x0a] enabled)
> (XEN) Processor #10 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x0c] enabled)
> (XEN) Processor #12 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x0e] enabled)
> (XEN) Processor #14 6:6 APIC version 21
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x06] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x07] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ERST table was not found
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
> (XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
> (XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
> (XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
> (XEN) Not enabling x2APIC: depends on iommu_supports_eim.
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
> (XEN) CPU0: No MCE banks present. Machine check support disabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2592.575 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
> (XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
> (XEN) PCI: MCFG area at f0000000 reserved in E820
> (XEN) PCI: Using MCFG for segment 0000 bus 00-7f
> (XEN) I/O virtualisation disabled
> (XEN) smp_store_cpu_info id=0
> (XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) set_nr_sockets nr_sockets=8 num_processors=8 disabled_cpus=0 cpus=1
> (XEN) Getting VERSION: 50015
> (XEN) Getting VERSION: 50015
> (XEN) Getting ID: 0
> (XEN) Getting LVT0: 700
> (XEN) Getting LVT1: 400
> (XEN) enabled ExtINT on CPU#0
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) init IO_APIC IRQs
> (XEN) IO-APIC (apicid-pin) 1-0, 1-16, 1-17, 1-18, 1-19, 1-20, 1-21, 1-22,
> 1-23 not connected.
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) number of MP IRQ sources: 15.
> (XEN) number of IO-APIC #1 registers: 24.
> (XEN) testing the IO APIC.......................
> (XEN) IO APIC #1......
> (XEN) .... register #00: 01000000
> (XEN) ....... : physical APIC id: 01
> (XEN) ....... : Delivery Type: 0
> (XEN) ....... : LTS : 0
> (XEN) .... register #01: 00170011
> (XEN) ....... : max redirection entries: 0017
> (XEN) ....... : PRQ implemented: 0
> (XEN) ....... : IO APIC version: 0011
> (XEN) .... register #02: 01000000
> (XEN) ....... : arbitration: 01
> (XEN) .... IRQ redirection table:
> (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
> (XEN) 00 000 00 1 0 0 0 0 0 0 00
> (XEN) 01 001 01 0 0 0 0 0 1 1 28
> (XEN) 02 001 01 0 0 0 0 0 1 1 F0
> (XEN) 03 001 01 0 0 0 0 0 1 1 30
> (XEN) 04 001 01 0 0 0 0 0 1 1 F1
> (XEN) 05 001 01 0 0 0 0 0 1 1 38
> (XEN) 06 001 01 0 0 0 0 0 1 1 40
> (XEN) 07 001 01 0 0 0 0 0 1 1 48
> (XEN) 08 001 01 0 0 0 0 0 1 1 50
> (XEN) 09 001 01 0 0 0 0 0 1 1 58
> (XEN) 0a 001 01 0 0 0 0 0 1 1 60
> (XEN) 0b 001 01 0 0 0 0 0 1 1 68
> (XEN) 0c 001 01 0 0 0 0 0 1 1 70
> (XEN) 0d 001 01 0 0 0 0 0 1 1 78
> (XEN) 0e 001 01 0 0 0 0 0 1 1 88
> (XEN) 0f 001 01 0 0 0 0 0 1 1 90
> (XEN) 10 000 00 1 0 0 0 0 0 0 00
> (XEN) 11 000 00 1 0 0 0 0 0 0 00
> (XEN) 12 000 00 1 0 0 0 0 0 0 00
> (XEN) 13 000 00 1 0 0 0 0 0 0 00
> (XEN) 14 000 00 1 0 0 0 0 0 0 00
> (XEN) 15 000 00 1 0 0 0 0 0 0 00
> (XEN) 16 000 00 1 0 0 0 0 0 0 00
> (XEN) 17 000 00 1 0 0 0 0 0 0 00
> (XEN) Using vector-based indexing
> (XEN) IRQ to pin mappings:
> (XEN) IRQ240 -> 0:2
> (XEN) IRQ40 -> 0:1
> (XEN) IRQ48 -> 0:3
> (XEN) IRQ241 -> 0:4
> (XEN) IRQ56 -> 0:5
> (XEN) IRQ64 -> 0:6
> (XEN) IRQ72 -> 0:7
> (XEN) IRQ80 -> 0:8
> (XEN) IRQ88 -> 0:9
> (XEN) IRQ96 -> 0:10
> (XEN) IRQ104 -> 0:11
> (XEN) IRQ112 -> 0:12
> (XEN) IRQ120 -> 0:13
> (XEN) IRQ136 -> 0:14
> (XEN) IRQ144 -> 0:15
> (XEN) .................................... done.
> (XEN) Using local APIC timer interrupts.
> (XEN) calibrating APIC timer ...
> (XEN) ..... CPU clock speed is 2592.9336 MHz.
> (XEN) ..... host bus clock speed is 92.6030 MHz.
> (XEN) ..... bus_scale = 0x5ed5
> (XEN) Platform timer is 3.579MHz ACPI PM Timer
> (XEN) Allocated console ring of 64 KiB.
> (XEN) mwait-idle: MWAIT substates: 0x10
> (XEN) mwait-idle: v0.4 model 0x46
> (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC TPR shadow
> (XEN) - Extended Page Tables (EPT)
> (XEN) - Virtual-Processor Identifiers (VPID)
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) not detected
> (XEN) HVM: PVH mode not supported on this platform
> (XEN) CPU 0 APIC 0 -> Node 0
> (XEN) CPU 1 APIC 2 -> Node 0
> (XEN) Booting processor 1/2 eip 8f000
> (XEN) Initializing CPU#1
> (XEN) masked ExtINT on CPU#1
> (XEN) smp_store_cpu_info id=1
> (XEN) CPU: Physical Processor ID: 2
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU1: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=1 socket=2
> socket_cpumask[socket]=0000000000000000
> (XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 2 APIC 4 -> Node 0
> (XEN) Booting processor 2/4 eip 8f000
> (XEN) Initializing CPU#2
> (XEN) masked ExtINT on CPU#2
> (XEN) smp_store_cpu_info id=2
> (XEN) CPU: Physical Processor ID: 4
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU2: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=2 socket=4
> socket_cpumask[socket]=0000000000000000
> (XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 3 APIC 6 -> Node 0
> (XEN) Booting processor 3/6 eip 8f000
> (XEN) Initializing CPU#3
> (XEN) masked ExtINT on CPU#3
> (XEN) smp_store_cpu_info id=3
> (XEN) CPU: Physical Processor ID: 6
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU3: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=3 socket=6
> socket_cpumask[socket]=0000000000000000
> (XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 4 APIC 8 -> Node 0
> (XEN) Booting processor 4/8 eip 8f000
> (XEN) Initializing CPU#4
> (XEN) masked ExtINT on CPU#4
> (XEN) smp_store_cpu_info id=4
> (XEN) CPU: Physical Processor ID: 8
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU4: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=4 socket=8
> socket_cpumask[socket]=ffff8301bd865430
> (XEN) CPU4: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 5 APIC 10 -> Node 0
> (XEN) Booting processor 5/10 eip 8f000
> (XEN) Initializing CPU#5
> (XEN) masked ExtINT on CPU#5
> (XEN) smp_store_cpu_info id=5
> (XEN) CPU: Physical Processor ID: 10
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU5: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=5 socket=10
> socket_cpumask[socket]=0000000000000001
> (XEN) CPU5: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) ----[ Xen-4.6.1-pre x86_64 debug=y Not tainted ]----
> (XEN) CPU: 5
> (XEN) RIP: e008:[<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
> (XEN) RFLAGS: 0000000000010093 CONTEXT: hypervisor
> (XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx: 000000313d571c80
> (XEN) rdx: 000000000000000a rsi: 0000000000000026 rdi: 0000000000000005
> (XEN) rbp: ffff8301bd8a7ee0 rsp: ffff8301bd8a7ea0 r8: ffff8301bd8a7ea0
> (XEN) r9: 0000000000000000 r10: 0000000000000001 r11: 0000000000000001
> (XEN) r12: 00000000000000c8 r13: 0000000000000005 r14: 0000000000000000
> (XEN) r15: 0000000000000000 cr0: 000000008005003b cr4: 00000000001526a0
> (XEN) cr3: 00000000bfc9f000 cr2: 0000000000000001
> (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
> (XEN) Xen stack trace from rsp=ffff8301bd8a7ea0:
> (XEN) 0000000500000000 0000000000000005 ffff8301bd8a7ee0
> 0000000000000000
> (XEN) 00000000000000c8 0000000000000005 0000000000000000
> 0000000000000000
> (XEN) ffff8301bd8a7f10 ffff82d080188fc3 0000000000000001
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000000 0000000000000000 0000000000000000
> 0000000000000000
> (XEN) 0000000000000005 ffff8300bfff9000 000000313d571c80
> 0000000000000000
> (XEN) Xen call trace:
> (XEN) [<ffff82d080188544>] set_cpu_sibling_map+0x53/0x38d
> (XEN) [<ffff82d080188fc3>] start_secondary+0x1fb/0x256
> (XEN)
> (XEN) Pagetable walk from 0000000000000001:
> (XEN) L4[0x000] = 00000001bd8ec063 ffffffffffffffff
> (XEN) L3[0x000] = 00000001bd8eb063 ffffffffffffffff
> (XEN) L2[0x000] = 00000001bd8ea063 ffffffffffffffff
> (XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
> (XEN)
> (XEN) ****************************************
> (XEN) Panic on CPU 5:
> (XEN) FATAL PAGE FAULT
> (XEN) [error_code=0002]
> (XEN) Faulting linear address: 0000000000000001
> (XEN) ****************************************
> (XEN)
> (XEN) Reboot in five seconds...
> (XEN) Resetting with ACPI MEMORY or I/O RESET_REG.
>
> With 3 CPUs the situation is completely different. nr_sockets=1, but
> socket is always 0 in smp_store_cpu_info().
>
> Loading xen-4.6-amd64.gz... ok
> Loading vmlinuz-3.14.51-grsec-dock... ok
> Loading initrd.img-3.14.51-grsec-dock... ok
> (XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
> eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov
> 20 10:07:47 PST 2015
> (XEN) Bootloader: SYSLINUX 4.05 20140113
> (XEN) Command line: console=com1,vga com1=115200 no-bootscrub
> dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f800 (usable)
> (XEN) 000000000009f800 - 00000000000a0000 (reserved)
> (XEN) 00000000000dc000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bfef0000 (usable)
> (XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
> (XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
> (XEN) 00000000bff00000 - 00000000c0000000 (usable)
> (XEN) 00000000f0000000 - 00000000f8000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee01000 (reserved)
> (XEN) 00000000fffe0000 - 0000000100000000 (reserved)
> (XEN) 0000000100000000 - 00000001c0000000 (usable)
> (XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
> (XEN) ACPI: XSDT BFEF0396, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
> (XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
> (XEN) ACPI: DSDT BFEF061E, E855 (r1 PTLTD Custom 6040000 MSFT 3000001)
> (XEN) ACPI: FACS BFEFFFC0, 0040
> (XEN) ACPI: BOOT BFEF05F6, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
> (XEN) ACPI: APIC BFEF058A, 006C (r1 PTLTD APIC 6040000 LTP 0)
> (XEN) ACPI: MCFG BFEF054E, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
> (XEN) ACPI: SRAT BFEF044E, 0100 (r2 VMWARE MEMPLUG 6040000 VMW 1)
> (XEN) ACPI: WAET BFEF0426, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
> (XEN) System RAM: 6143MB (6291004kB)
> (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
> (XEN) SRAT: Node 0 PXM 0 0-a0000
> (XEN) SRAT: Node 0 PXM 0 100000-10000000
> (XEN) SRAT: Node 0 PXM 0 10000000-c0000000
> (XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
> (XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
> (XEN) NUMA: Using 8 for the hash shift.
> (XEN) Domain heap initialised
> (XEN) found SMP MP-table at 000f6a80
> (XEN) DMI present.
> (XEN) APIC boot state is 'xapic'
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
> (XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> (XEN) Processor #2 6:6 APIC version 21
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x03] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 3, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ERST table was not found
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) SMP: Allowing 3 CPUs (0 hotplug CPUs)
> (XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
> (XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
> (XEN) IRQ limits: 24 GSI, 568 MSI/MSI-X
> (XEN) Not enabling x2APIC: depends on iommu_supports_eim.
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 0
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
> (XEN) CPU0: No MCE banks present. Machine check support disabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2592.635 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
> (XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
> (XEN) PCI: MCFG area at f0000000 reserved in E820
> (XEN) PCI: Using MCFG for segment 0000 bus 00-7f
> (XEN) I/O virtualisation disabled
> (XEN) smp_store_cpu_info id=0
> (XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) set_nr_sockets nr_sockets=1 num_processors=3 disabled_cpus=0 cpus=3
> (XEN) Getting VERSION: 50015
> (XEN) Getting VERSION: 50015
> (XEN) Getting ID: 0
> (XEN) Getting LVT0: 700
> (XEN) Getting LVT1: 400
> (XEN) enabled ExtINT on CPU#0
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) init IO_APIC IRQs
> (XEN) IO-APIC (apicid-pin) 3-0, 3-16, 3-17, 3-18, 3-19, 3-20, 3-21, 3-22,
> 3-23 not connected.
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) number of MP IRQ sources: 15.
> (XEN) number of IO-APIC #3 registers: 24.
> (XEN) testing the IO APIC.......................
> (XEN) IO APIC #3......
> (XEN) .... register #00: 03000000
> (XEN) ....... : physical APIC id: 03
> (XEN) ....... : Delivery Type: 0
> (XEN) ....... : LTS : 0
> (XEN) .... register #01: 00170011
> (XEN) ....... : max redirection entries: 0017
> (XEN) ....... : PRQ implemented: 0
> (XEN) ....... : IO APIC version: 0011
> (XEN) .... register #02: 03000000
> (XEN) ....... : arbitration: 03
> (XEN) .... IRQ redirection table:
> (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
> (XEN) 00 000 00 1 0 0 0 0 0 0 00
> (XEN) 01 001 01 0 0 0 0 0 1 1 28
> (XEN) 02 001 01 0 0 0 0 0 1 1 F0
> (XEN) 03 001 01 0 0 0 0 0 1 1 30
> (XEN) 04 001 01 0 0 0 0 0 1 1 F1
> (XEN) 05 001 01 0 0 0 0 0 1 1 38
> (XEN) 06 001 01 0 0 0 0 0 1 1 40
> (XEN) 07 001 01 0 0 0 0 0 1 1 48
> (XEN) 08 001 01 0 0 0 0 0 1 1 50
> (XEN) 09 001 01 0 0 0 0 0 1 1 58
> (XEN) 0a 001 01 0 0 0 0 0 1 1 60
> (XEN) 0b 001 01 0 0 0 0 0 1 1 68
> (XEN) 0c 001 01 0 0 0 0 0 1 1 70
> (XEN) 0d 001 01 0 0 0 0 0 1 1 78
> (XEN) 0e 001 01 0 0 0 0 0 1 1 88
> (XEN) 0f 001 01 0 0 0 0 0 1 1 90
> (XEN) 10 000 00 1 0 0 0 0 0 0 00
> (XEN) 11 000 00 1 0 0 0 0 0 0 00
> (XEN) 12 000 00 1 0 0 0 0 0 0 00
> (XEN) 13 000 00 1 0 0 0 0 0 0 00
> (XEN) 14 000 00 1 0 0 0 0 0 0 00
> (XEN) 15 000 00 1 0 0 0 0 0 0 00
> (XEN) 16 000 00 1 0 0 0 0 0 0 00
> (XEN) 17 000 00 1 0 0 0 0 0 0 00
> (XEN) Using vector-based indexing
> (XEN) IRQ to pin mappings:
> (XEN) IRQ240 -> 0:2
> (XEN) IRQ40 -> 0:1
> (XEN) IRQ48 -> 0:3
> (XEN) IRQ241 -> 0:4
> (XEN) IRQ56 -> 0:5
> (XEN) IRQ64 -> 0:6
> (XEN) IRQ72 -> 0:7
> (XEN) IRQ80 -> 0:8
> (XEN) IRQ88 -> 0:9
> (XEN) IRQ96 -> 0:10
> (XEN) IRQ104 -> 0:11
> (XEN) IRQ112 -> 0:12
> (XEN) IRQ120 -> 0:13
> (XEN) IRQ136 -> 0:14
> (XEN) IRQ144 -> 0:15
> (XEN) .................................... done.
> (XEN) Using local APIC timer interrupts.
> (XEN) calibrating APIC timer ...
> (XEN) ..... CPU clock speed is 2592.9209 MHz.
> (XEN) ..... host bus clock speed is 92.5952 MHz.
> (XEN) ..... bus_scale = 0x5ed2
> (XEN) Platform timer is 3.579MHz ACPI PM Timer
> (XEN) Allocated console ring of 32 KiB.
> (XEN) mwait-idle: MWAIT substates: 0x10
> (XEN) mwait-idle: v0.4 model 0x46
> (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC TPR shadow
> (XEN) - Extended Page Tables (EPT)
> (XEN) - Virtual-Processor Identifiers (VPID)
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) not detected
> (XEN) HVM: PVH mode not supported on this platform
> (XEN) CPU 0 APIC 0 -> Node 0
> (XEN) CPU 1 APIC 1 -> Node 0
> (XEN) Booting processor 1/1 eip 8f000
> (XEN) Initializing CPU#1
> (XEN) masked ExtINT on CPU#1
> (XEN) smp_store_cpu_info id=1
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 1
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU1: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=1 socket=0
> socket_cpumask[socket]=ffff8301bd8e5460
> (XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 2 APIC 2 -> Node 0
> (XEN) Booting processor 2/2 eip 8f000
> (XEN) Initializing CPU#2
> (XEN) masked ExtINT on CPU#2
> (XEN) smp_store_cpu_info id=2
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 2
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU2: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=2 socket=0
> socket_cpumask[socket]=ffff8301bd8e5460
> (XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) Brought up 3 CPUs
> (XEN) ACPI sleep modes: S3
> (XEN) VPMU: disabled
> (XEN) Dom0 has maximum 408 PIRQs
> (XEN) NX (Execute Disable) protection active
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x68f6d8
> (XEN) elf_parse_binary: phdr: paddr=0x1690000 memsz=0x409000
> (XEN) elf_parse_binary: phdr: paddr=0x1a99000 memsz=0x32c120
> (XEN) elf_parse_binary: phdr: paddr=0x1dc6000 memsz=0x1000
> (XEN) elf_parse_binary: phdr: paddr=0x1dc7000 memsz=0x1d13c0
> (XEN) elf_parse_binary: phdr: paddr=0x1f99000 memsz=0x53000
> (XEN) elf_parse_binary: phdr: paddr=0x1fec000 memsz=0x1250
> (XEN) elf_parse_binary: phdr: paddr=0x1fee000 memsz=0x1012000
> (XEN) elf_parse_binary: memory: 0x1000000 -> 0x3000000
> (XEN) elf_xen_parse_note: GUEST_OS = "linux"
> (XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
> (XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
> (XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
> (XEN) elf_xen_parse_note: ENTRY = 0xffffffff81f991f0
> (XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
> (XEN) elf_xen_parse_note: FEATURES =
> "!writable_page_tables|pae_pgdir_above_4gb"
> (XEN) elf_xen_parse_note: SUPPORTED_FEATURES = 0x801
> (XEN) elf_xen_parse_note: PAE_MODE = "yes"
> (XEN) elf_xen_parse_note: LOADER = "generic"
> (XEN) elf_xen_parse_note: unknown xen elf note (0xd)
> (XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
> (XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
> (XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
> (XEN) elf_xen_addr_calc_check: addresses:
> (XEN) virt_base = 0xffffffff80000000
> (XEN) elf_paddr_offset = 0x0
> (XEN) virt_offset = 0xffffffff80000000
> (XEN) virt_kstart = 0xffffffff81000000
> (XEN) virt_kend = 0xffffffff83000000
> (XEN) virt_entry = 0xffffffff81f991f0
> (XEN) p2m_base = 0xffffffffffffffff
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
> be allocated)
> (XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
> (XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
> (XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
> (XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
> (XEN) Page tables: ffffffff8412d000->ffffffff84152000
> (XEN) Boot stack: ffffffff84152000->ffffffff84153000
> (XEN) TOTAL: ffffffff80000000->ffffffff84400000
> (XEN) ENTRY ADDRESS: ffffffff81f991f0
> (XEN) Dom0 has maximum 3 VCPUs
> (XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8168f6d8
> (XEN) elf_load_binary: phdr 1 at 0xffffffff81690000 -> 0xffffffff81a99000
> (XEN) elf_load_binary: phdr 2 at 0xffffffff81a99000 -> 0xffffffff81dc5120
> (XEN) elf_load_binary: phdr 3 at 0xffffffff81dc6000 -> 0xffffffff81dc7000
> (XEN) elf_load_binary: phdr 4 at 0xffffffff81dc7000 -> 0xffffffff81f983c0
> (XEN) elf_load_binary: phdr 5 at 0xffffffff81f99000 -> 0xffffffff81fec000
> (XEN) elf_load_binary: phdr 6 at 0xffffffff81fec000 -> 0xffffffff81fed250
> (XEN) elf_load_binary: phdr 7 at 0xffffffff81fee000 -> 0xffffffff820ce000
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
> to Xen)
> (XEN) Freed 308kB init memory.
> mapping kernel into physical memory
> about to get started...
>
> Similarly with 6 CPUs, nr_sockets=2, and socket is always 0 or 1 in
> smp_store_cpu_info().
>
> Loading xen-4.6-amd64.gz... ok
> Loading vmlinuz-3.14.51-grsec-dock... ok
> Loading initrd.img-3.14.51-grsec-dock... ok
> (XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
> eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Fri Nov
> 20 10:07:47 PST 2015
> (XEN) Bootloader: SYSLINUX 4.05 20140113
> (XEN) Command line: console=com1,vga com1=115200 no-bootscrub
> dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
> (XEN) Video information:
> (XEN) VGA is text mode 80x25, font 8x16
> (XEN) Disc information:
> (XEN) Found 1 MBR signatures
> (XEN) Found 1 EDD information structures
> (XEN) Xen-e820 RAM map:
> (XEN) 0000000000000000 - 000000000009f800 (usable)
> (XEN) 000000000009f800 - 00000000000a0000 (reserved)
> (XEN) 00000000000dc000 - 0000000000100000 (reserved)
> (XEN) 0000000000100000 - 00000000bfef0000 (usable)
> (XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
> (XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
> (XEN) 00000000bff00000 - 00000000c0000000 (usable)
> (XEN) 00000000f0000000 - 00000000f8000000 (reserved)
> (XEN) 00000000fec00000 - 00000000fec10000 (reserved)
> (XEN) 00000000fee00000 - 00000000fee01000 (reserved)
> (XEN) 00000000fffe0000 - 0000000100000000 (reserved)
> (XEN) 0000000100000000 - 00000001c0000000 (usable)
> (XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
> (XEN) ACPI: XSDT BFEF01F9, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
> (XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
> (XEN) ACPI: DSDT BFEF04DB, E998 (r1 PTLTD Custom 6040000 MSFT 3000001)
> (XEN) ACPI: FACS BFEFFFC0, 0040
> (XEN) ACPI: BOOT BFEF04B3, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
> (XEN) ACPI: APIC BFEF041D, 0096 (r1 PTLTD APIC 6040000 LTP 0)
> (XEN) ACPI: MCFG BFEF03E1, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
> (XEN) ACPI: SRAT BFEF02B1, 0130 (r2 VMWARE MEMPLUG 6040000 VMW 1)
> (XEN) ACPI: WAET BFEF0289, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
> (XEN) System RAM: 6143MB (6291004kB)
> (XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 01 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 05 -> Node 0
> (XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
> (XEN) SRAT: Node 0 PXM 0 0-a0000
> (XEN) SRAT: Node 0 PXM 0 100000-10000000
> (XEN) SRAT: Node 0 PXM 0 10000000-c0000000
> (XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
> (XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
> (XEN) NUMA: Using 8 for the hash shift.
> (XEN) Domain heap initialised
> (XEN) found SMP MP-table at 000f6a80
> (XEN) DMI present.
> (XEN) APIC boot state is 'xapic'
> (XEN) Using APIC driver default
> (XEN) ACPI: PM-Timer IO Port: 0x1008
> (XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
> (XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
> (XEN) ACPI: Local APIC address 0xfee00000
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
> (XEN) Processor #1 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
> (XEN) Processor #2 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
> (XEN) Processor #4 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x05] enabled)
> (XEN) Processor #5 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x06] enabled)
> (XEN) Processor #6 6:6 APIC version 21
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x04] high edge lint[0x1])
> (XEN) ACPI: LAPIC_NMI (acpi_id[0x05] high edge lint[0x1])
> (XEN) ACPI: IOAPIC (id[0x03] address[0xfec00000] gsi_base[0])
> (XEN) IOAPIC[0]: apic_id 3, version 17, address 0xfec00000, GSI 0-23
> (XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
> (XEN) ACPI: IRQ0 used by override.
> (XEN) ACPI: IRQ2 used by override.
> (XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
> (XEN) ERST table was not found
> (XEN) Using ACPI (MADT) for SMP configuration information
> (XEN) SMP: Allowing 6 CPUs (0 hotplug CPUs)
> (XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
> (XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
> (XEN) IRQ limits: 24 GSI, 1144 MSI/MSI-X
> (XEN) Not enabling x2APIC: depends on iommu_supports_eim.
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 0
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
> (XEN) CPU0: No MCE banks present. Machine check support disabled
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Initializing CPU#0
> (XEN) Detected 2592.566 MHz processor.
> (XEN) Initing memory sharing.
> (XEN) alt table ffff82d0802e6f10 -> ffff82d0802e81c4
> (XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
> (XEN) PCI: MCFG area at f0000000 reserved in E820
> (XEN) PCI: Using MCFG for segment 0000 bus 00-7f
> (XEN) I/O virtualisation disabled
> (XEN) smp_store_cpu_info id=0
> (XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) set_nr_sockets nr_sockets=2 num_processors=6 disabled_cpus=0 cpus=3
> (XEN) Getting VERSION: 50015
> (XEN) Getting VERSION: 50015
> (XEN) Getting ID: 0
> (XEN) Getting LVT0: 700
> (XEN) Getting LVT1: 400
> (XEN) enabled ExtINT on CPU#0
> (XEN) ENABLING IO-APIC IRQs
> (XEN) -> Using new ACK method
> (XEN) init IO_APIC IRQs
> (XEN) IO-APIC (apicid-pin) 3-0, 3-16, 3-17, 3-18, 3-19, 3-20, 3-21, 3-22,
> 3-23 not connected.
> (XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
> (XEN) number of MP IRQ sources: 15.
> (XEN) number of IO-APIC #3 registers: 24.
> (XEN) testing the IO APIC.......................
> (XEN) IO APIC #3......
> (XEN) .... register #00: 03000000
> (XEN) ....... : physical APIC id: 03
> (XEN) ....... : Delivery Type: 0
> (XEN) ....... : LTS : 0
> (XEN) .... register #01: 00170011
> (XEN) ....... : max redirection entries: 0017
> (XEN) ....... : PRQ implemented: 0
> (XEN) ....... : IO APIC version: 0011
> (XEN) .... register #02: 03000000
> (XEN) ....... : arbitration: 03
> (XEN) .... IRQ redirection table:
> (XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
> (XEN) 00 000 00 1 0 0 0 0 0 0 00
> (XEN) 01 001 01 0 0 0 0 0 1 1 28
> (XEN) 02 001 01 0 0 0 0 0 1 1 F0
> (XEN) 03 001 01 0 0 0 0 0 1 1 30
> (XEN) 04 001 01 0 0 0 0 0 1 1 F1
> (XEN) 05 001 01 0 0 0 0 0 1 1 38
> (XEN) 06 001 01 0 0 0 0 0 1 1 40
> (XEN) 07 001 01 0 0 0 0 0 1 1 48
> (XEN) 08 001 01 0 0 0 0 0 1 1 50
> (XEN) 09 001 01 0 0 0 0 0 1 1 58
> (XEN) 0a 001 01 0 0 0 0 0 1 1 60
> (XEN) 0b 001 01 0 0 0 0 0 1 1 68
> (XEN) 0c 001 01 0 0 0 0 0 1 1 70
> (XEN) 0d 001 01 0 0 0 0 0 1 1 78
> (XEN) 0e 001 01 0 0 0 0 0 1 1 88
> (XEN) 0f 001 01 0 0 0 0 0 1 1 90
> (XEN) 10 000 00 1 0 0 0 0 0 0 00
> (XEN) 11 000 00 1 0 0 0 0 0 0 00
> (XEN) 12 000 00 1 0 0 0 0 0 0 00
> (XEN) 13 000 00 1 0 0 0 0 0 0 00
> (XEN) 14 000 00 1 0 0 0 0 0 0 00
> (XEN) 15 000 00 1 0 0 0 0 0 0 00
> (XEN) 16 000 00 1 0 0 0 0 0 0 00
> (XEN) 17 000 00 1 0 0 0 0 0 0 00
> (XEN) Using vector-based indexing
> (XEN) IRQ to pin mappings:
> (XEN) IRQ240 -> 0:2
> (XEN) IRQ40 -> 0:1
> (XEN) IRQ48 -> 0:3
> (XEN) IRQ241 -> 0:4
> (XEN) IRQ56 -> 0:5
> (XEN) IRQ64 -> 0:6
> (XEN) IRQ72 -> 0:7
> (XEN) IRQ80 -> 0:8
> (XEN) IRQ88 -> 0:9
> (XEN) IRQ96 -> 0:10
> (XEN) IRQ104 -> 0:11
> (XEN) IRQ112 -> 0:12
> (XEN) IRQ120 -> 0:13
> (XEN) IRQ136 -> 0:14
> (XEN) IRQ144 -> 0:15
> (XEN) .................................... done.
> (XEN) Using local APIC timer interrupts.
> (XEN) calibrating APIC timer ...
> (XEN) ..... CPU clock speed is 2592.9460 MHz.
> (XEN) ..... host bus clock speed is 92.5859 MHz.
> (XEN) ..... bus_scale = 0x5ed0
> (XEN) Platform timer is 3.579MHz ACPI PM Timer
> (XEN) Allocated console ring of 64 KiB.
> (XEN) mwait-idle: MWAIT substates: 0x10
> (XEN) mwait-idle: v0.4 model 0x46
> (XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
> (XEN) VMX: Supported advanced features:
> (XEN) - APIC TPR shadow
> (XEN) - Extended Page Tables (EPT)
> (XEN) - Virtual-Processor Identifiers (VPID)
> (XEN) - Virtual NMI
> (XEN) - MSR direct-access bitmap
> (XEN) - Unrestricted Guest
> (XEN) HVM: ASIDs enabled.
> (XEN) HVM: VMX enabled
> (XEN) HVM: Hardware Assisted Paging (HAP) not detected
> (XEN) HVM: PVH mode not supported on this platform
> (XEN) CPU 0 APIC 0 -> Node 0
> (XEN) CPU 1 APIC 1 -> Node 0
> (XEN) Booting processor 1/1 eip 8f000
> (XEN) Initializing CPU#1
> (XEN) masked ExtINT on CPU#1
> (XEN) smp_store_cpu_info id=1
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 1
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU1: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=1 socket=0
> socket_cpumask[socket]=ffff8301bd84d460
> (XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 2 APIC 2 -> Node 0
> (XEN) Booting processor 2/2 eip 8f000
> (XEN) Initializing CPU#2
> (XEN) masked ExtINT on CPU#2
> (XEN) smp_store_cpu_info id=2
> (XEN) CPU: Physical Processor ID: 0
> (XEN) CPU: Processor Core ID: 2
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU2: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=2 socket=0
> socket_cpumask[socket]=ffff8301bd84d460
> (XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 3 APIC 4 -> Node 0
> (XEN) Booting processor 3/4 eip 8f000
> (XEN) Initializing CPU#3
> (XEN) masked ExtINT on CPU#3
> (XEN) smp_store_cpu_info id=3
> (XEN) CPU: Physical Processor ID: 1
> (XEN) CPU: Processor Core ID: 0
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU3: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=3 socket=1
> socket_cpumask[socket]=0000000000000000
> (XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 4 APIC 5 -> Node 0
> (XEN) Booting processor 4/5 eip 8f000
> (XEN) Initializing CPU#4
> (XEN) masked ExtINT on CPU#4
> (XEN) smp_store_cpu_info id=4
> (XEN) CPU: Physical Processor ID: 1
> (XEN) CPU: Processor Core ID: 1
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU4: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=4 socket=1
> socket_cpumask[socket]=ffff8301bd84df90
> (XEN) CPU4: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) CPU 5 APIC 6 -> Node 0
> (XEN) Booting processor 5/6 eip 8f000
> (XEN) Initializing CPU#5
> (XEN) masked ExtINT on CPU#5
> (XEN) smp_store_cpu_info id=5
> (XEN) CPU: Physical Processor ID: 1
> (XEN) CPU: Processor Core ID: 2
> (XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
> (XEN) CPU: L2 cache: 256K
> (XEN) CPU: L3 cache: 6144K
> (XEN) CPU5: No MCE banks present. Machine check support disabled
> (XEN) smp_store_cpu_info id=5 socket=1
> socket_cpumask[socket]=ffff8301bd84df90
> (XEN) CPU5: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
> (XEN) Brought up 6 CPUs
> (XEN) ACPI sleep modes: S3
> (XEN) VPMU: disabled
> (XEN) Dom0 has maximum 600 PIRQs
> (XEN) NX (Execute Disable) protection active
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x68f6d8
> (XEN) elf_parse_binary: phdr: paddr=0x1690000 memsz=0x409000
> (XEN) elf_parse_binary: phdr: paddr=0x1a99000 memsz=0x32c120
> (XEN) elf_parse_binary: phdr: paddr=0x1dc6000 memsz=0x1000
> (XEN) elf_parse_binary: phdr: paddr=0x1dc7000 memsz=0x1d13c0
> (XEN) elf_parse_binary: phdr: paddr=0x1f99000 memsz=0x53000
> (XEN) elf_parse_binary: phdr: paddr=0x1fec000 memsz=0x1250
> (XEN) elf_parse_binary: phdr: paddr=0x1fee000 memsz=0x1012000
> (XEN) elf_parse_binary: memory: 0x1000000 -> 0x3000000
> (XEN) elf_xen_parse_note: GUEST_OS = "linux"
> (XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
> (XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
> (XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
> (XEN) elf_xen_parse_note: ENTRY = 0xffffffff81f991f0
> (XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
> (XEN) elf_xen_parse_note: FEATURES =
> "!writable_page_tables|pae_pgdir_above_4gb"
> (XEN) elf_xen_parse_note: SUPPORTED_FEATURES = 0x801
> (XEN) elf_xen_parse_note: PAE_MODE = "yes"
> (XEN) elf_xen_parse_note: LOADER = "generic"
> (XEN) elf_xen_parse_note: unknown xen elf note (0xd)
> (XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
> (XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
> (XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
> (XEN) elf_xen_addr_calc_check: addresses:
> (XEN) virt_base = 0xffffffff80000000
> (XEN) elf_paddr_offset = 0x0
> (XEN) virt_offset = 0xffffffff80000000
> (XEN) virt_kstart = 0xffffffff81000000
> (XEN) virt_kend = 0xffffffff83000000
> (XEN) virt_entry = 0xffffffff81f991f0
> (XEN) p2m_base = 0xffffffffffffffff
> (XEN) Xen kernel: 64-bit, lsb, compat32
> (XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
> (XEN) PHYSICAL MEMORY ARRANGEMENT:
> (XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
> be allocated)
> (XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
> (XEN) VIRTUAL MEMORY ARRANGEMENT:
> (XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
> (XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
> (XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
> (XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
> (XEN) Page tables: ffffffff8412d000->ffffffff84152000
> (XEN) Boot stack: ffffffff84152000->ffffffff84153000
> (XEN) TOTAL: ffffffff80000000->ffffffff84400000
> (XEN) ENTRY ADDRESS: ffffffff81f991f0
> (XEN) Dom0 has maximum 6 VCPUs
> (XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8168f6d8
> (XEN) elf_load_binary: phdr 1 at 0xffffffff81690000 -> 0xffffffff81a99000
> (XEN) elf_load_binary: phdr 2 at 0xffffffff81a99000 -> 0xffffffff81dc5120
> (XEN) elf_load_binary: phdr 3 at 0xffffffff81dc6000 -> 0xffffffff81dc7000
> (XEN) elf_load_binary: phdr 4 at 0xffffffff81dc7000 -> 0xffffffff81f983c0
> (XEN) elf_load_binary: phdr 5 at 0xffffffff81f99000 -> 0xffffffff81fec000
> (XEN) elf_load_binary: phdr 6 at 0xffffffff81fec000 -> 0xffffffff81fed250
> (XEN) elf_load_binary: phdr 7 at 0xffffffff81fee000 -> 0xffffffff820ce000
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) Xen is relinquishing VGA console.
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
> to Xen)
> (XEN) Freed 308kB init memory.
> mapping kernel into physical memory
> about to get started...
>
> So it looks like for power-of-2 CPUs, Fusion is providing a separate
> socket per CPU, numbered 0, 2, 4, ...
>
> For multiple-of-3 CPUs, Fusion is providing a socket for every 3 CPUs,
> numbered 0, 1, ...
>
> --Ed
>
>
> On Fri, Nov 20, 2015 at 2:25 AM, Jan Beulich <JBeulich@suse.com> wrote:
>
>> >>> On 20.11.15 at 02:22, <eswierk@skyportsystems.com> wrote:
>> > (XEN) ----[ Xen-4.6.1-pre x86_64 debug=n Not tainted ]----
>> > (XEN) CPU: 3
>> > (XEN) RIP: e008:[<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
>> > (XEN) RFLAGS: 0000000000010006 CONTEXT: hypervisor
>> > (XEN) rax: 0000000000000001 rbx: 0000000000000000 rcx:
>> 000000313d5b4080
>> > (XEN) rdx: 0000000000000006 rsi: 0000000000000000 rdi:
>> 0000000000000003
>> > (XEN) rbp: 0000000000000300 rsp: ffff8301bd87fe90 r8:
>> ffff8301bd878000
>> > (XEN) r9: 000000313d5b4080 r10: 0000000000000001 r11:
>> 0000000000000001
>> > (XEN) r12: ffff82d0802fd500 r13: 0000000000000000 r14:
>> 0000000000000000
>> > (XEN) r15: 0000000000000003 cr0: 000000008005003b cr4:
>> 00000000001526a0
>> > (XEN) cr3: 00000000bfc75000 cr2: 0000000000000001
>> > (XEN) ds: 0000 es: 0000 fs: 0000 gs: 0000 ss: 0000 cs: e008
>> > (XEN) Xen stack trace from rsp=ffff8301bd87fe90:
>> > (XEN) 00000003802fd800 0000000000000018 0000000000000000
>> 0000010000000000
>> > (XEN) ffff82d0802fd800 0000000000000000 00000000000000c8
>> 0000000000000003
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> ffff82d0801834dc
>> > (XEN) 0000000000000000 0000000000000001 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000000
>> 0000000000000000
>> > (XEN) 0000000000000000 0000000000000000 0000000000000003
>> ffff8300bfafc000
>> > (XEN) 000000313d5b4080 0000000000000000
>> > (XEN) Xen call trace:
>> > (XEN) [<ffff82d08018302f>] set_cpu_sibling_map+0x3f/0x330
>> > (XEN) [<ffff82d0801834dc>] start_secondary+0x1bc/0x250
>> > (XEN)
>> > (XEN) Pagetable walk from 0000000000000001:
>> > (XEN) L4[0x000] = 00000001bd8f0063 ffffffffffffffff
>> > (XEN) L3[0x000] = 00000001bd8ef063 ffffffffffffffff
>> > (XEN) L2[0x000] = 00000001bd8ee063 ffffffffffffffff
>> > (XEN) L1[0x000] = 0000000000000000 ffffffffffffffff
>> > (XEN)
>> > (XEN) ****************************************
>> > (XEN) Panic on CPU 3:
>> > (XEN) FATAL PAGE FAULT
>> > (XEN) [error_code=0002]
>> > (XEN) Faulting linear address: 0000000000000001
>> > (XEN) ****************************************
>> > (XEN)
>> > (XEN) Reboot in five seconds...
>> >
>> > set_cpu_sibling_map+0x3f is the second cpumask_set_cpu() call in
>> > set_cpu_sibling_map():
>> >
>> http://xenbits.xen.org/gitweb/?p=xen.git;a=blob;f=xen/arch/x86/smpboot.c;h=0
>> >
>> 94699286f4f6962942024ec8b2b24c7b7996cc0;hb=78833c04250416f1870c458309d3ac0e5c
>> > f915fd#l261
>>
>> I suppose cpu_to_socket(cpu) returns a value for which the
>> socket_cpumask[] entry didn't get set up yet. But to prove that,
>> we'd need to see the disassembly around the code location
>> above, to be able to associate register values with variables.
>>
>> If that's the case, then I'd further guess that the CPUID
>> information provided by Fusion isn't exactly as one would expect
>> on real hardware. Whether we need to fix something, or can
>> work around a quirk of theirs depends on the exact nature of
>> the issue. Instrumenting code populating socket_cpumask[]
>> would be a good first step.
>>
>> Jan
>>
>>
>
[-- Attachment #1.2: Type: text/html, Size: 78723 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-21 1:21 ` Ed Swierk
@ 2015-11-23 1:10 ` Chao Peng
2015-11-23 5:39 ` Chao Peng
2015-11-23 10:18 ` Jan Beulich
1 sibling, 1 reply; 22+ messages in thread
From: Chao Peng @ 2015-11-23 1:10 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, Jan Beulich, xen-devel
On Fri, Nov 20, 2015 at 05:21:11PM -0800, Ed Swierk wrote:
> The problem is that the index of the socket_cpumask array is derived via
> cpu_to_socket() from the APIC ID of the processor in a given socket, but
> the size of the array is computed based on nr_sockets, which is not
> necessarily equal to the maximum APIC ID.
>
> Sizing the socket_cpumask to MAX_APICS rather than nr_sockets seems safer,
> though a bit wasteful. I verified that this change fixes the boot crash
> with 4 or 8 CPUs on VMware Fusion.
>
> --- a/xen/arch/x86/smpboot.c
> +++ b/xen/arch/x86/smpboot.c
> @@ -819,7 +819,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
>
> set_nr_sockets();
>
> - socket_cpumask = xzalloc_array(cpumask_t *, nr_sockets);
> + socket_cpumask = xzalloc_array(cpumask_t *, MAX_APICS);
Just replacing nr_sockets with MAX_APICS can not really solve problem.
socket_cpumask should always be synchronized with nr_sockets, otherwise
at least some function will be missing, if not cause panic in another
place.
If possible, I'd suggest you can debug set_nr_sockets(), especially you
can inspect the following two values for panic case:
boot_cpu_data.x86_max_cores
boot_cpu_data.x86_num_siblings
Thanks,
Chao
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-23 1:10 ` Chao Peng
@ 2015-11-23 5:39 ` Chao Peng
0 siblings, 0 replies; 22+ messages in thread
From: Chao Peng @ 2015-11-23 5:39 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, Jan Beulich, xen-devel
On Mon, Nov 23, 2015 at 09:10:08AM +0800, Chao Peng wrote:
> On Fri, Nov 20, 2015 at 05:21:11PM -0800, Ed Swierk wrote:
> > The problem is that the index of the socket_cpumask array is derived via
> > cpu_to_socket() from the APIC ID of the processor in a given socket, but
> > the size of the array is computed based on nr_sockets, which is not
> > necessarily equal to the maximum APIC ID.
> >
> > Sizing the socket_cpumask to MAX_APICS rather than nr_sockets seems safer,
> > though a bit wasteful. I verified that this change fixes the boot crash
> > with 4 or 8 CPUs on VMware Fusion.
> >
> > --- a/xen/arch/x86/smpboot.c
> > +++ b/xen/arch/x86/smpboot.c
> > @@ -819,7 +819,7 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
> >
> > set_nr_sockets();
> >
> > - socket_cpumask = xzalloc_array(cpumask_t *, nr_sockets);
> > + socket_cpumask = xzalloc_array(cpumask_t *, MAX_APICS);
>
> Just replacing nr_sockets with MAX_APICS can not really solve problem.
> socket_cpumask should always be synchronized with nr_sockets, otherwise
> at least some function will be missing, if not cause panic in another
> place.
>
> If possible, I'd suggest you can debug set_nr_sockets(), especially you
> can inspect the following two values for panic case:
> boot_cpu_data.x86_max_cores
> boot_cpu_data.x86_num_siblings
After carefully checked the log, it looks nr_sockets is computed
correctly for your case, instead phys_proc_id is not right. It could be
again caused by bad CPUID information. Therefor you need debug the CPU
detection code which set phys_proc_id.
Thanks,
Chao
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-21 1:21 ` Ed Swierk
2015-11-23 1:10 ` Chao Peng
@ 2015-11-23 10:18 ` Jan Beulich
2015-11-23 16:36 ` Ed Swierk
1 sibling, 1 reply; 22+ messages in thread
From: Jan Beulich @ 2015-11-23 10:18 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, xen-devel, Chao Peng
>>> On 21.11.15 at 02:21, <eswierk@skyportsystems.com> wrote:
> The problem is that the index of the socket_cpumask array is derived via
> cpu_to_socket() from the APIC ID of the processor in a given socket, but
> the size of the array is computed based on nr_sockets, which is not
> necessarily equal to the maximum APIC ID.
>
> Sizing the socket_cpumask to MAX_APICS rather than nr_sockets seems safer,
> though a bit wasteful. I verified that this change fixes the boot crash
> with 4 or 8 CPUs on VMware Fusion.
But that raises the question of sanity of the CPUID output Xen gets
presented: With
> (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> (XEN) Processor #0 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> (XEN) Processor #2 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> (XEN) Processor #4 6:6 APIC version 21
> (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> (XEN) Processor #6 6:6 APIC version 21
and taking the output you added, I can only suspect that the value
used for determining the socket shift is unexpected (CPUID leaf 0xb).
Could you supply the observed values? (See
detect_extended_topology() and set_nr_sockets().) As you can
see, the core IDs ("CPU: Physical Processor ID: ...") aren't sequential,
which we expect them to be (with holes left only when non-power-of-2
values need taking care of).
Jan
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-23 10:18 ` Jan Beulich
@ 2015-11-23 16:36 ` Ed Swierk
2015-11-24 10:34 ` Jan Beulich
0 siblings, 1 reply; 22+ messages in thread
From: Ed Swierk @ 2015-11-23 16:36 UTC (permalink / raw)
To: Jan Beulich; +Cc: Andrew Cooper, xen-devel, Chao Peng
[-- Attachment #1.1: Type: text/plain, Size: 17066 bytes --]
I instrumented detect_extended_topology() and ran again with 4 CPUs.
Loading xen-4.6-amd64.gz... ok
Loading vmlinuz-3.14.51-grsec-dock... ok
Loading initrd.img-3.14.51-grsec-dock... ok
(XEN) Xen version 4.6.1-pre (Debian 4.6.1~pre-1skyport1) (
eswierk@skyportsystems.com) (gcc (Debian 4.9.3-4) 4.9.3) debug=y Mon Nov 23
07:18:36 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1 apic_verbosity=debug
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bd8f8000 - 1bd8fa000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) APIC boot state is 'xapic'
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) mapped APIC to ffff82cfffdfb000 (fee00000)
(XEN) mapped IOAPIC to ffff82cfffdfa000 (fec00000)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x0
(XEN) detect_extended_topology initial_apicid=0 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x0
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.632 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802e7f90 -> ffff82d0802e9244
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) smp_store_cpu_info id=0
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) set_nr_sockets nr_sockets=4 num_processors=4 disabled_cpus=0 cpus=1
(XEN) x86_max_cores=1 x86_num_siblings=1
(XEN) first=0
(XEN) weight=4
(XEN) cpu 0
(XEN) cpu 2
(XEN) cpu 4
(XEN) cpu 6
(XEN) Getting VERSION: 50015
(XEN) Getting VERSION: 50015
(XEN) Getting ID: 0
(XEN) Getting LVT0: 700
(XEN) Getting LVT1: 400
(XEN) enabled ExtINT on CPU#0
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) init IO_APIC IRQs
(XEN) IO-APIC (apicid-pin) 1-0, 1-16, 1-17, 1-18, 1-19, 1-20, 1-21, 1-22,
1-23 not connected.
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) number of MP IRQ sources: 15.
(XEN) number of IO-APIC #1 registers: 24.
(XEN) testing the IO APIC.......................
(XEN) IO APIC #1......
(XEN) .... register #00: 01000000
(XEN) ....... : physical APIC id: 01
(XEN) ....... : Delivery Type: 0
(XEN) ....... : LTS : 0
(XEN) .... register #01: 00170011
(XEN) ....... : max redirection entries: 0017
(XEN) ....... : PRQ implemented: 0
(XEN) ....... : IO APIC version: 0011
(XEN) .... register #02: 01000000
(XEN) ....... : arbitration: 01
(XEN) .... IRQ redirection table:
(XEN) NR Log Phy Mask Trig IRR Pol Stat Dest Deli Vect:
(XEN) 00 000 00 1 0 0 0 0 0 0 00
(XEN) 01 001 01 0 0 0 0 0 1 1 28
(XEN) 02 001 01 0 0 0 0 0 1 1 F0
(XEN) 03 001 01 0 0 0 0 0 1 1 30
(XEN) 04 001 01 0 0 0 0 0 1 1 F1
(XEN) 05 001 01 0 0 0 0 0 1 1 38
(XEN) 06 001 01 0 0 0 0 0 1 1 40
(XEN) 07 001 01 0 0 0 0 0 1 1 48
(XEN) 08 001 01 0 0 0 0 0 1 1 50
(XEN) 09 001 01 0 0 0 0 0 1 1 58
(XEN) 0a 001 01 0 0 0 0 0 1 1 60
(XEN) 0b 001 01 0 0 0 0 0 1 1 68
(XEN) 0c 001 01 0 0 0 0 0 1 1 70
(XEN) 0d 001 01 0 0 0 0 0 1 1 78
(XEN) 0e 001 01 0 0 0 0 0 1 1 88
(XEN) 0f 001 01 0 0 0 0 0 1 1 90
(XEN) 10 000 00 1 0 0 0 0 0 0 00
(XEN) 11 000 00 1 0 0 0 0 0 0 00
(XEN) 12 000 00 1 0 0 0 0 0 0 00
(XEN) 13 000 00 1 0 0 0 0 0 0 00
(XEN) 14 000 00 1 0 0 0 0 0 0 00
(XEN) 15 000 00 1 0 0 0 0 0 0 00
(XEN) 16 000 00 1 0 0 0 0 0 0 00
(XEN) 17 000 00 1 0 0 0 0 0 0 00
(XEN) Using vector-based indexing
(XEN) IRQ to pin mappings:
(XEN) IRQ240 -> 0:2
(XEN) IRQ40 -> 0:1
(XEN) IRQ48 -> 0:3
(XEN) IRQ241 -> 0:4
(XEN) IRQ56 -> 0:5
(XEN) IRQ64 -> 0:6
(XEN) IRQ72 -> 0:7
(XEN) IRQ80 -> 0:8
(XEN) IRQ88 -> 0:9
(XEN) IRQ96 -> 0:10
(XEN) IRQ104 -> 0:11
(XEN) IRQ112 -> 0:12
(XEN) IRQ120 -> 0:13
(XEN) IRQ136 -> 0:14
(XEN) IRQ144 -> 0:15
(XEN) .................................... done.
(XEN) Using local APIC timer interrupts.
(XEN) calibrating APIC timer ...
(XEN) ..... CPU clock speed is 2592.9214 MHz.
(XEN) ..... host bus clock speed is 92.5796 MHz.
(XEN) ..... bus_scale = 0x5ece
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) masked ExtINT on CPU#1
(XEN) smp_store_cpu_info id=1
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x2
(XEN) detect_extended_topology initial_apicid=2 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x2
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=1 socket=2
socket_cpumask[socket]=0000000000000000
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) masked ExtINT on CPU#2
(XEN) smp_store_cpu_info id=2
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x4
(XEN) detect_extended_topology initial_apicid=4 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x4
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=2 socket=4
socket_cpumask[socket]=0000000000000000
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 6 -> Node 0
(XEN) Booting processor 3/6 eip 8f000
(XEN) Initializing CPU#3
(XEN) masked ExtINT on CPU#3
(XEN) smp_store_cpu_info id=3
(XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1
ecx=0x100 edx=0x6
(XEN) detect_extended_topology initial_apicid=6 core_plus_mask_width=0
core_level_siblings=1
(XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1
ecx=0x201 edx=0x6
(XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0
core_select_mask=0x0 core_level_siblings=1
(XEN) CPU: Physical Processor ID: 6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) smp_store_cpu_info id=3 socket=6
socket_cpumask[socket]=0000000000000000
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) Brought up 4 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) Dom0 has maximum 600 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) elf_parse_binary: phdr: paddr=0x1000000 memsz=0x68f6d8
(XEN) elf_parse_binary: phdr: paddr=0x1690000 memsz=0x409000
(XEN) elf_parse_binary: phdr: paddr=0x1a99000 memsz=0x32c120
(XEN) elf_parse_binary: phdr: paddr=0x1dc6000 memsz=0x1000
(XEN) elf_parse_binary: phdr: paddr=0x1dc7000 memsz=0x1d13c0
(XEN) elf_parse_binary: phdr: paddr=0x1f99000 memsz=0x53000
(XEN) elf_parse_binary: phdr: paddr=0x1fec000 memsz=0x1250
(XEN) elf_parse_binary: phdr: paddr=0x1fee000 memsz=0x1012000
(XEN) elf_parse_binary: memory: 0x1000000 -> 0x3000000
(XEN) elf_xen_parse_note: GUEST_OS = "linux"
(XEN) elf_xen_parse_note: GUEST_VERSION = "2.6"
(XEN) elf_xen_parse_note: XEN_VERSION = "xen-3.0"
(XEN) elf_xen_parse_note: VIRT_BASE = 0xffffffff80000000
(XEN) elf_xen_parse_note: ENTRY = 0xffffffff81f991f0
(XEN) elf_xen_parse_note: HYPERCALL_PAGE = 0xffffffff81001000
(XEN) elf_xen_parse_note: FEATURES =
"!writable_page_tables|pae_pgdir_above_4gb"
(XEN) elf_xen_parse_note: SUPPORTED_FEATURES = 0x801
(XEN) elf_xen_parse_note: PAE_MODE = "yes"
(XEN) elf_xen_parse_note: LOADER = "generic"
(XEN) elf_xen_parse_note: unknown xen elf note (0xd)
(XEN) elf_xen_parse_note: SUSPEND_CANCEL = 0x1
(XEN) elf_xen_parse_note: HV_START_LOW = 0xffff800000000000
(XEN) elf_xen_parse_note: PADDR_OFFSET = 0x0
(XEN) elf_xen_addr_calc_check: addresses:
(XEN) virt_base = 0xffffffff80000000
(XEN) elf_paddr_offset = 0x0
(XEN) virt_offset = 0xffffffff80000000
(XEN) virt_kstart = 0xffffffff81000000
(XEN) virt_kend = 0xffffffff83000000
(XEN) virt_entry = 0xffffffff81f991f0
(XEN) p2m_base = 0xffffffffffffffff
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x3000000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504532 pages to
be allocated)
(XEN) Init. ramdisk: 00000001bf2d4000->00000001bffff800
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff83000000
(XEN) Init. ramdisk: ffffffff83000000->ffffffff83d2b800
(XEN) Phys-Mach map: ffffffff83d2c000->ffffffff8412c000
(XEN) Start info: ffffffff8412c000->ffffffff8412c4b4
(XEN) Page tables: ffffffff8412d000->ffffffff84152000
(XEN) Boot stack: ffffffff84152000->ffffffff84153000
(XEN) TOTAL: ffffffff80000000->ffffffff84400000
(XEN) ENTRY ADDRESS: ffffffff81f991f0
(XEN) Dom0 has maximum 4 VCPUs
(XEN) elf_load_binary: phdr 0 at 0xffffffff81000000 -> 0xffffffff8168f6d8
(XEN) elf_load_binary: phdr 1 at 0xffffffff81690000 -> 0xffffffff81a99000
(XEN) elf_load_binary: phdr 2 at 0xffffffff81a99000 -> 0xffffffff81dc5120
(XEN) elf_load_binary: phdr 3 at 0xffffffff81dc6000 -> 0xffffffff81dc7000
(XEN) elf_load_binary: phdr 4 at 0xffffffff81dc7000 -> 0xffffffff81f983c0
(XEN) elf_load_binary: phdr 5 at 0xffffffff81f99000 -> 0xffffffff81fec000
(XEN) elf_load_binary: phdr 6 at 0xffffffff81fec000 -> 0xffffffff81fed250
(XEN) elf_load_binary: phdr 7 at 0xffffffff81fee000 -> 0xffffffff820ce000
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input
to Xen)
(XEN) Freed 304kB init memory.
mapping kernel into physical memory
about to get started...
If cpuid 0xb returned 1 rather than 0 in eax[4:0], we would get
consecutively-numbered physical processor IDs.
But the only requirement I see in the IA SDM (vol 2A, table 3-17) is that
the eax[4:0] value yield unique IDs, not necessarily consecutive. Likewise
while the examples in vol 3A sec 8.9 show physical IDs numbered
consecutively, the algorithms do not assume this is the case.
--Ed
On Mon, Nov 23, 2015 at 2:18 AM, Jan Beulich <JBeulich@suse.com> wrote:
> >>> On 21.11.15 at 02:21, <eswierk@skyportsystems.com> wrote:
> > The problem is that the index of the socket_cpumask array is derived via
> > cpu_to_socket() from the APIC ID of the processor in a given socket, but
> > the size of the array is computed based on nr_sockets, which is not
> > necessarily equal to the maximum APIC ID.
> >
> > Sizing the socket_cpumask to MAX_APICS rather than nr_sockets seems
> safer,
> > though a bit wasteful. I verified that this change fixes the boot crash
> > with 4 or 8 CPUs on VMware Fusion.
>
> But that raises the question of sanity of the CPUID output Xen gets
> presented: With
>
> > (XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
> > (XEN) Processor #0 6:6 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
> > (XEN) Processor #2 6:6 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
> > (XEN) Processor #4 6:6 APIC version 21
> > (XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
> > (XEN) Processor #6 6:6 APIC version 21
>
> and taking the output you added, I can only suspect that the value
> used for determining the socket shift is unexpected (CPUID leaf 0xb).
> Could you supply the observed values? (See
> detect_extended_topology() and set_nr_sockets().) As you can
> see, the core IDs ("CPU: Physical Processor ID: ...") aren't sequential,
> which we expect them to be (with holes left only when non-power-of-2
> values need taking care of).
>
> Jan
>
>
[-- Attachment #1.2: Type: text/html, Size: 21744 bytes --]
[-- Attachment #2: Type: text/plain, Size: 126 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-23 16:36 ` Ed Swierk
@ 2015-11-24 10:34 ` Jan Beulich
2015-11-24 14:13 ` Ed Swierk
2015-11-25 7:48 ` Chao Peng
0 siblings, 2 replies; 22+ messages in thread
From: Jan Beulich @ 2015-11-24 10:34 UTC (permalink / raw)
To: Chao Peng, Ed Swierk; +Cc: Andrew Cooper, xen-devel
>>> On 23.11.15 at 17:36, <eswierk@skyportsystems.com> wrote:
> I instrumented detect_extended_topology() and ran again with 4 CPUs.
>[...]
> (XEN) smp_store_cpu_info id=3
> (XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1 ecx=0x100 edx=0x6
> (XEN) detect_extended_topology initial_apicid=6 core_plus_mask_width=0 core_level_siblings=1
> (XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1 ecx=0x201 edx=0x6
> (XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0 core_select_mask=0x0 core_level_siblings=1
>[...]
> If cpuid 0xb returned 1 rather than 0 in eax[4:0], we would get
> consecutively-numbered physical processor IDs.
>
> But the only requirement I see in the IA SDM (vol 2A, table 3-17) is that
> the eax[4:0] value yield unique IDs, not necessarily consecutive. Likewise
> while the examples in vol 3A sec 8.9 show physical IDs numbered
> consecutively, the algorithms do not assume this is the case.
Indeed, and I think I had said so. The algorithm does, however, tell
us that with the above output CPU 3 (APIC ID 6) is on socket 6 (both
shifts being zero), which for the whole system results in sockets 1,
3, and 5 unused. While not explicitly excluded, I'm not sure how far
we should go in expecting all kinds of odd configurations (along those
lines we e.g. have a limit on the largest APIC ID we allow: MAX_APICS /
MAX_LOCAL_APIC, which for big systems is 4 times the number of
CPUs we support).
Taking it to set_nr_sockets(), a pretty basic assumption is broken by
the above way of presenting topology: We would have to have more
sockets than there are CPUs. I would have wanted to check what
e.g. Linux does here, but there doesn't seem to be any support of
CAT (and hence any need for per-socket data) there.
(I am, btw, now also confused by you saying that e.g. for a 3-CPU
config things work. If the topology data gets presented in similar
ways in that case, I can't see why you wouldn't run into the same
problem. Unless memory corruption occurs silently in one case, but
"loudly" in the other.)
Bottom line - for the moment I do not see a reasonable way of
dealing with that situation. The closest I could see would be what
we iirc had temporarily during the review cycles of the initial CAT
series: A command line option to specify the number of sockets. Or
make all accesses to socket_cpumask[] conditional upon PSR being
enabled (which would have the bad side effect of making future
uses for other purposes more cumbersome), or go through and
range check the socket number on all of those accesses.
Chao, could you - inside Intel - please check whether there are
any assumptions on the respective CPUID leaf output that aren't
explicitly stated in the SDM right now (like resulting in contiguous
socket numbers), and ask for them getting made explicit (if there
are any), or it being made explicit that no assumptions at all are
to be made at all on the presented values (in which case we'd
have to consume MADT parsing data in set_nr_sockets(), e.g.
by replacing num_processors there with one more than the
maximum APIC ID of any non-disabled CPU)?
Jan
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-24 10:34 ` Jan Beulich
@ 2015-11-24 14:13 ` Ed Swierk
2015-11-24 15:20 ` Jan Beulich
2015-11-25 7:48 ` Chao Peng
1 sibling, 1 reply; 22+ messages in thread
From: Ed Swierk @ 2015-11-24 14:13 UTC (permalink / raw)
To: Jan Beulich; +Cc: Chao Peng, xen-devel, Andrew Cooper
On Tue, Nov 24, 2015 at 2:34 AM, Jan Beulich <JBeulich@suse.com> wrote:
> Indeed, and I think I had said so. The algorithm does, however, tell
> us that with the above output CPU 3 (APIC ID 6) is on socket 6 (both
> shifts being zero), which for the whole system results in sockets 1,
> 3, and 5 unused. While not explicitly excluded, I'm not sure how far
> we should go in expecting all kinds of odd configurations (along those
> lines we e.g. have a limit on the largest APIC ID we allow: MAX_APICS /
> MAX_LOCAL_APIC, which for big systems is 4 times the number of
> CPUs we support).
That's why I thought it reasonable to substitute MAX_APICS for
nr_sockets in sizing the socket_cpumask array.
> Taking it to set_nr_sockets(), a pretty basic assumption is broken by
> the above way of presenting topology: We would have to have more
> sockets than there are CPUs. I would have wanted to check what
> e.g. Linux does here, but there doesn't seem to be any support of
> CAT (and hence any need for per-socket data) there.
I looked at Linux, and there is no per-socket bookkeeping, AFAICT.
> (I am, btw, now also confused by you saying that e.g. for a 3-CPU
> config things work. If the topology data gets presented in similar
> ways in that case, I can't see why you wouldn't run into the same
> problem. Unless memory corruption occurs silently in one case, but
> "loudly" in the other.)
For 3, 6 and 12 CPUs, Fusion presents a completely different topology,
with 3-core sockets numbered consecutively starting with 0.
> Bottom line - for the moment I do not see a reasonable way of
> dealing with that situation. The closest I could see would be what
> we iirc had temporarily during the review cycles of the initial CAT
> series: A command line option to specify the number of sockets. Or
> make all accesses to socket_cpumask[] conditional upon PSR being
> enabled (which would have the bad side effect of making future
> uses for other purposes more cumbersome), or go through and
> range check the socket number on all of those accesses.
Could we avoid the issue by replacing socket_cpumask array with a list
or hashtable, indexed by socket ID?
> Chao, could you - inside Intel - please check whether there are
> any assumptions on the respective CPUID leaf output that aren't
> explicitly stated in the SDM right now (like resulting in contiguous
> socket numbers), and ask for them getting made explicit (if there
> are any), or it being made explicit that no assumptions at all are
> to be made at all on the presented values (in which case we'd
> have to consume MADT parsing data in set_nr_sockets(), e.g.
> by replacing num_processors there with one more than the
> maximum APIC ID of any non-disabled CPU)?
I suppose the key is whether Intel has encoded such assumptions in the
BIOS reference code, or has otherwise communicated them to AMI et al.
--Ed
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-24 14:13 ` Ed Swierk
@ 2015-11-24 15:20 ` Jan Beulich
2015-11-24 20:28 ` Ed Swierk
0 siblings, 1 reply; 22+ messages in thread
From: Jan Beulich @ 2015-11-24 15:20 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, xen-devel, Chao Peng
>>> On 24.11.15 at 15:13, <eswierk@skyportsystems.com> wrote:
> On Tue, Nov 24, 2015 at 2:34 AM, Jan Beulich <JBeulich@suse.com> wrote:
>> Bottom line - for the moment I do not see a reasonable way of
>> dealing with that situation. The closest I could see would be what
>> we iirc had temporarily during the review cycles of the initial CAT
>> series: A command line option to specify the number of sockets. Or
>> make all accesses to socket_cpumask[] conditional upon PSR being
>> enabled (which would have the bad side effect of making future
>> uses for other purposes more cumbersome), or go through and
>> range check the socket number on all of those accesses.
>
> Could we avoid the issue by replacing socket_cpumask array with a list
> or hashtable, indexed by socket ID?
Yes, a radix tree would work. But it would also seem like overkill if
all we need it for is some strange virtualization of CPUID. The more
I think about it, the better I like the option below.
Jan
>> Chao, could you - inside Intel - please check whether there are
>> any assumptions on the respective CPUID leaf output that aren't
>> explicitly stated in the SDM right now (like resulting in contiguous
>> socket numbers), and ask for them getting made explicit (if there
>> are any), or it being made explicit that no assumptions at all are
>> to be made at all on the presented values (in which case we'd
>> have to consume MADT parsing data in set_nr_sockets(), e.g.
>> by replacing num_processors there with one more than the
>> maximum APIC ID of any non-disabled CPU)?
>
> I suppose the key is whether Intel has encoded such assumptions in the
> BIOS reference code, or has otherwise communicated them to AMI et al.
>
> --Ed
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-24 15:20 ` Jan Beulich
@ 2015-11-24 20:28 ` Ed Swierk
2015-11-25 9:11 ` Jan Beulich
2015-12-02 13:48 ` Jan Beulich
0 siblings, 2 replies; 22+ messages in thread
From: Ed Swierk @ 2015-11-24 20:28 UTC (permalink / raw)
To: Jan Beulich; +Cc: Andrew Cooper, xen-devel, Chao Peng
RFC. Boot tested on VMware Fusion, and on a 2-socket Xeon server.
diff --git a/xen/include/asm-x86/smp.h b/xen/include/asm-x86/smp.h
index ea07888..a41ce2d 100644
--- a/xen/include/asm-x86/smp.h
+++ b/xen/include/asm-x86/smp.h
@@ -67,7 +67,7 @@ extern unsigned int nr_sockets;
void set_nr_sockets(void);
/* Representing HT and core siblings in each socket. */
-extern cpumask_t **socket_cpumask;
+cpumask_t *socket_cpumask(unsigned int socket);
#endif /* !__ASSEMBLY__ */
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 0946992..6aadaac 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -60,7 +60,7 @@ cpumask_t cpu_online_map __read_mostly;
EXPORT_SYMBOL(cpu_online_map);
unsigned int __read_mostly nr_sockets;
-cpumask_t **__read_mostly socket_cpumask;
+static struct radix_tree_root socket_cpumask_tree;
static cpumask_t *secondary_socket_cpumask;
struct cpuinfo_x86 cpu_data[NR_CPUS];
@@ -81,6 +81,11 @@ static enum cpu_state {
void *stack_base[NR_CPUS];
+cpumask_t *socket_cpumask(unsigned int socket)
+{
+ return radix_tree_lookup(&socket_cpumask_tree, socket);
+}
+
static void smp_store_cpu_info(int id)
{
struct cpuinfo_x86 *c = cpu_data + id;
@@ -92,9 +97,9 @@ static void smp_store_cpu_info(int id)
identify_cpu(c);
socket = cpu_to_socket(id);
- if ( !socket_cpumask[socket] )
+ if ( radix_tree_insert(&socket_cpumask_tree, socket,
+ secondary_socket_cpumask) == 0 )
{
- socket_cpumask[socket] = secondary_socket_cpumask;
secondary_socket_cpumask = NULL;
}
}
@@ -258,7 +263,7 @@ static void set_cpu_sibling_map(int cpu)
cpumask_set_cpu(cpu, &cpu_sibling_setup_map);
- cpumask_set_cpu(cpu, socket_cpumask[cpu_to_socket(cpu)]);
+ cpumask_set_cpu(cpu, socket_cpumask(cpu_to_socket(cpu)));
if ( c[cpu].x86_num_siblings > 1 )
{
@@ -666,11 +671,12 @@ static void cpu_smpboot_free(unsigned int cpu)
{
unsigned int order, socket = cpu_to_socket(cpu);
struct cpuinfo_x86 *c = cpu_data;
+ cpumask_t *m = socket_cpumask(socket);
- if ( cpumask_empty(socket_cpumask[socket]) )
+ if ( m && cpumask_empty(m) )
{
- xfree(socket_cpumask[socket]);
- socket_cpumask[socket] = NULL;
+ radix_tree_delete(&socket_cpumask_tree, socket);
+ xfree(m);
}
c[cpu].phys_proc_id = XEN_INVALID_SOCKET_ID;
@@ -804,6 +810,8 @@ static struct notifier_block cpu_smpboot_nfb = {
void __init smp_prepare_cpus(unsigned int max_cpus)
{
+ cpumask_t *m;
+
register_cpu_notifier(&cpu_smpboot_nfb);
mtrr_aps_sync_begin();
@@ -819,9 +827,9 @@ void __init smp_prepare_cpus(unsigned int max_cpus)
set_nr_sockets();
- socket_cpumask = xzalloc_array(cpumask_t *, nr_sockets);
- if ( socket_cpumask == NULL ||
- (socket_cpumask[cpu_to_socket(0)] = xzalloc(cpumask_t)) == NULL )
+ radix_tree_init(&socket_cpumask_tree);
+ if ( (m = xzalloc(cpumask_t)) == NULL ||
+ radix_tree_insert(&socket_cpumask_tree, cpu_to_socket(0), m) != 0 )
panic("No memory for socket CPU siblings map");
if ( !zalloc_cpumask_var(&per_cpu(cpu_sibling_mask, 0)) ||
@@ -888,7 +896,7 @@ remove_siblinginfo(int cpu)
{
int sibling;
- cpumask_clear_cpu(cpu, socket_cpumask[cpu_to_socket(cpu)]);
+ cpumask_clear_cpu(cpu, socket_cpumask(cpu_to_socket(cpu)));
for_each_cpu ( sibling, per_cpu(cpu_core_mask, cpu) )
{
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index c0daa2e..7acb3d9 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -52,14 +52,6 @@ static DEFINE_PER_CPU(struct psr_assoc, psr_assoc);
static struct psr_cat_cbm *temp_cos_to_cbm;
-static unsigned int get_socket_cpu(unsigned int socket)
-{
- if ( likely(socket < nr_sockets) )
- return cpumask_any(socket_cpumask[socket]);
-
- return nr_cpu_ids;
-}
-
static void __init parse_psr_bool(char *s, char *value, char *feature,
unsigned int mask)
{
@@ -331,7 +323,8 @@ static int write_l3_cbm(unsigned int socket,
unsigned int cos, uint64_t cbm)
do_write_l3_cbm(&info);
else
{
- unsigned int cpu = get_socket_cpu(socket);
+ cpumask_t *m = socket_cpumask(socket);
+ unsigned int cpu = m ? cpumask_any(m) : nr_cpu_ids;
if ( cpu >= nr_cpu_ids )
return -ENOTSOCK;
@@ -503,8 +496,9 @@ static void cat_cpu_init(void)
static void cat_cpu_fini(unsigned int cpu)
{
unsigned int socket = cpu_to_socket(cpu);
+ cpumask_t *m = socket_cpumask(socket);
- if ( !socket_cpumask[socket] || cpumask_empty(socket_cpumask[socket]) )
+ if ( !m || cpumask_empty(m) )
{
struct psr_cat_socket_info *info = cat_socket_info + socket;
On Tue, Nov 24, 2015 at 7:20 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 24.11.15 at 15:13, <eswierk@skyportsystems.com> wrote:
>> On Tue, Nov 24, 2015 at 2:34 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>> Bottom line - for the moment I do not see a reasonable way of
>>> dealing with that situation. The closest I could see would be what
>>> we iirc had temporarily during the review cycles of the initial CAT
>>> series: A command line option to specify the number of sockets. Or
>>> make all accesses to socket_cpumask[] conditional upon PSR being
>>> enabled (which would have the bad side effect of making future
>>> uses for other purposes more cumbersome), or go through and
>>> range check the socket number on all of those accesses.
>>
>> Could we avoid the issue by replacing socket_cpumask array with a list
>> or hashtable, indexed by socket ID?
>
> Yes, a radix tree would work. But it would also seem like overkill if
> all we need it for is some strange virtualization of CPUID. The more
> I think about it, the better I like the option below.
>
> Jan
>
>>> Chao, could you - inside Intel - please check whether there are
>>> any assumptions on the respective CPUID leaf output that aren't
>>> explicitly stated in the SDM right now (like resulting in contiguous
>>> socket numbers), and ask for them getting made explicit (if there
>>> are any), or it being made explicit that no assumptions at all are
>>> to be made at all on the presented values (in which case we'd
>>> have to consume MADT parsing data in set_nr_sockets(), e.g.
>>> by replacing num_processors there with one more than the
>>> maximum APIC ID of any non-disabled CPU)?
>>
>> I suppose the key is whether Intel has encoded such assumptions in the
>> BIOS reference code, or has otherwise communicated them to AMI et al.
>>
>> --Ed
>
>
>
^ permalink raw reply related [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-24 10:34 ` Jan Beulich
2015-11-24 14:13 ` Ed Swierk
@ 2015-11-25 7:48 ` Chao Peng
2015-11-25 9:04 ` Jan Beulich
1 sibling, 1 reply; 22+ messages in thread
From: Chao Peng @ 2015-11-25 7:48 UTC (permalink / raw)
To: Jan Beulich; +Cc: Andrew Cooper, Ed Swierk, xen-devel
On Tue, Nov 24, 2015 at 03:34:45AM -0700, Jan Beulich wrote:
> >>> On 23.11.15 at 17:36, <eswierk@skyportsystems.com> wrote:
> > I instrumented detect_extended_topology() and ran again with 4 CPUs.
> >[...]
> > (XEN) smp_store_cpu_info id=3
> > (XEN) detect_extended_topology cpuid_count op=0xb count=0 eax=0x0 ebx=0x1 ecx=0x100 edx=0x6
> > (XEN) detect_extended_topology initial_apicid=6 core_plus_mask_width=0 core_level_siblings=1
> > (XEN) detect_extended_topology cpuid_count op=0xb count=1 eax=0x0 ebx=0x1 ecx=0x201 edx=0x6
> > (XEN) detect_extended_topology ht_mask_width=0 core_plus_mask_width=0 core_select_mask=0x0 core_level_siblings=1
> >[...]
> > If cpuid 0xb returned 1 rather than 0 in eax[4:0], we would get
> > consecutively-numbered physical processor IDs.
> >
> > But the only requirement I see in the IA SDM (vol 2A, table 3-17) is that
> > the eax[4:0] value yield unique IDs, not necessarily consecutive. Likewise
> > while the examples in vol 3A sec 8.9 show physical IDs numbered
> > consecutively, the algorithms do not assume this is the case.
>
> Indeed, and I think I had said so. The algorithm does, however, tell
> us that with the above output CPU 3 (APIC ID 6) is on socket 6 (both
> shifts being zero), which for the whole system results in sockets 1,
> 3, and 5 unused. While not explicitly excluded, I'm not sure how far
> we should go in expecting all kinds of odd configurations (along those
> lines we e.g. have a limit on the largest APIC ID we allow: MAX_APICS /
> MAX_LOCAL_APIC, which for big systems is 4 times the number of
> CPUs we support).
>
> Taking it to set_nr_sockets(), a pretty basic assumption is broken by
> the above way of presenting topology: We would have to have more
> sockets than there are CPUs. I would have wanted to check what
> e.g. Linux does here, but there doesn't seem to be any support of
> CAT (and hence any need for per-socket data) there.
Actually I checked Linux code when I implementing this but it doesn't
exist. Current Linux CAT patch supports only system-level other than
per-socket level so it doesn't need that as well. There are people
requesting to add per-socket support so Linux need solve this problem
eventually. But at this time, we don't have any reference.
>
> (I am, btw, now also confused by you saying that e.g. for a 3-CPU
> config things work. If the topology data gets presented in similar
> ways in that case, I can't see why you wouldn't run into the same
> problem. Unless memory corruption occurs silently in one case, but
> "loudly" in the other.)
>
> Bottom line - for the moment I do not see a reasonable way of
> dealing with that situation. The closest I could see would be what
> we iirc had temporarily during the review cycles of the initial CAT
> series: A command line option to specify the number of sockets. Or
> make all accesses to socket_cpumask[] conditional upon PSR being
> enabled (which would have the bad side effect of making future
> uses for other purposes more cumbersome), or go through and
> range check the socket number on all of those accesses.
>
> Chao, could you - inside Intel - please check whether there are
> any assumptions on the respective CPUID leaf output that aren't
> explicitly stated in the SDM right now (like resulting in contiguous
> socket numbers), and ask for them getting made explicit (if there
> are any), or it being made explicit that no assumptions at all are
> to be made at all on the presented values
Actually there is already such statement in SDM (ch8.9.1, vol3):
"The value of valid APIC_IDs need not be contiguous across package
boundary or core boundaries".
> (in which case we'd
> have to consume MADT parsing data in set_nr_sockets(), e.g.
> by replacing num_processors there with one more than the
> maximum APIC ID of any non-disabled CPU)?
Even with this, we still have problem for hotplug case, the inserted
CPU may have a APIC_ID bigger than the maximum APIC_ID here.
But let's back to the real world. Most machines that support CAT should
have continuous SOCKET_ID so it's not a problem. Giving that CAT is the
only feature uses this, I guess this suggestion might be better than
other solutions in practice.
Chao
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-25 7:48 ` Chao Peng
@ 2015-11-25 9:04 ` Jan Beulich
2015-11-25 23:27 ` Ed Swierk
0 siblings, 1 reply; 22+ messages in thread
From: Jan Beulich @ 2015-11-25 9:04 UTC (permalink / raw)
To: Chao Peng; +Cc: Andrew Cooper, Ed Swierk, xen-devel
>>> On 25.11.15 at 08:48, <chao.p.peng@linux.intel.com> wrote:
> On Tue, Nov 24, 2015 at 03:34:45AM -0700, Jan Beulich wrote:
>> Chao, could you - inside Intel - please check whether there are
>> any assumptions on the respective CPUID leaf output that aren't
>> explicitly stated in the SDM right now (like resulting in contiguous
>> socket numbers), and ask for them getting made explicit (if there
>> are any), or it being made explicit that no assumptions at all are
>> to be made at all on the presented values
>
> Actually there is already such statement in SDM (ch8.9.1, vol3):
>
> "The value of valid APIC_IDs need not be contiguous across package
> boundary or core boundaries".
That's a statement on APIC ID space (which necessarily can't be
contiguous on systems with a non-power-of-2 core count), but I
was asking about the socket ID space.
>> (in which case we'd
>> have to consume MADT parsing data in set_nr_sockets(), e.g.
>> by replacing num_processors there with one more than the
>> maximum APIC ID of any non-disabled CPU)?
>
> Even with this, we still have problem for hotplug case, the inserted
> CPU may have a APIC_ID bigger than the maximum APIC_ID here.
>
> But let's back to the real world. Most machines that support CAT should
> have continuous SOCKET_ID so it's not a problem. Giving that CAT is the
> only feature uses this, I guess this suggestion might be better than
> other solutions in practice.
And we could actually cater for that by extrapolating the value
added to cover disabled_cpus.
Jan
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-24 20:28 ` Ed Swierk
@ 2015-11-25 9:11 ` Jan Beulich
2015-12-02 13:48 ` Jan Beulich
1 sibling, 0 replies; 22+ messages in thread
From: Jan Beulich @ 2015-11-25 9:11 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, xen-devel, Chao Peng
>>> On 24.11.15 at 21:28, <eswierk@skyportsystems.com> wrote:
> RFC. Boot tested on VMware Fusion, and on a 2-socket Xeon server.
Well, thanks, but as said I view this is overkill (and I'm also not sure
what you have is completely race free). Hence I'd prefer a more light
weight solution if at all possible.
Jan
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-25 9:04 ` Jan Beulich
@ 2015-11-25 23:27 ` Ed Swierk
2015-11-26 7:49 ` Jan Beulich
0 siblings, 1 reply; 22+ messages in thread
From: Ed Swierk @ 2015-11-25 23:27 UTC (permalink / raw)
To: Jan Beulich; +Cc: Chao Peng, xen-devel, Andrew Cooper
A few more data points: I also tested Xen 4.6 on VMware ESXi 5.5, and
it yields similar results. Not surprising, since Fusion uses basically
the same virtualization engine.
However, ESXi offers many more choices of number of processors, number
of cores, hyperthreading, etc. The weird processor ID assignment (0,
2, 4, 6, ...) occurs only with 4 or 8 processors, 1 core per socket,
and no hyperthreading. If I change any of these parameters, the
processor IDs become sequential.
It appears in the 4- and 8-processor cases, VMware is emulating
something like a Xeon E7340:
https://github.com/deater/test_proc/blob/master/x86_64/x86_64.intel.6.15.11.xeon_e7340
In fact someone asked a question about running Xen on this platform
way back when: http://lists.xenproject.org/archives/html/xen-users/2008-05/msg00691.html
Others of similar vintage assign processor IDs 0 and 3 on a
2-processor system:
https://www.centos.org/forums/viewtopic.php?t=30255
or even 0 and 6: http://serverfault.com/questions/302429/interpreting-cpuinfo
So there are real hardware platforms with non-sequential processor
IDs. They are quite ancient and don't support CAT, but that doesn't
rule out the possibility of a newer or future platform behaving
similarly.
At least there is no evidence of a platform assigning extremely large
processor IDs; until then we are safe using arrays and bitmaps. The
issue is sizing these data structures appropriately.
--Ed
On Wed, Nov 25, 2015 at 1:04 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 25.11.15 at 08:48, <chao.p.peng@linux.intel.com> wrote:
>> On Tue, Nov 24, 2015 at 03:34:45AM -0700, Jan Beulich wrote:
>>> Chao, could you - inside Intel - please check whether there are
>>> any assumptions on the respective CPUID leaf output that aren't
>>> explicitly stated in the SDM right now (like resulting in contiguous
>>> socket numbers), and ask for them getting made explicit (if there
>>> are any), or it being made explicit that no assumptions at all are
>>> to be made at all on the presented values
>>
>> Actually there is already such statement in SDM (ch8.9.1, vol3):
>>
>> "The value of valid APIC_IDs need not be contiguous across package
>> boundary or core boundaries".
>
> That's a statement on APIC ID space (which necessarily can't be
> contiguous on systems with a non-power-of-2 core count), but I
> was asking about the socket ID space.
>
>>> (in which case we'd
>>> have to consume MADT parsing data in set_nr_sockets(), e.g.
>>> by replacing num_processors there with one more than the
>>> maximum APIC ID of any non-disabled CPU)?
>>
>> Even with this, we still have problem for hotplug case, the inserted
>> CPU may have a APIC_ID bigger than the maximum APIC_ID here.
>>
>> But let's back to the real world. Most machines that support CAT should
>> have continuous SOCKET_ID so it's not a problem. Giving that CAT is the
>> only feature uses this, I guess this suggestion might be better than
>> other solutions in practice.
>
> And we could actually cater for that by extrapolating the value
> added to cover disabled_cpus.
>
> Jan
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-25 23:27 ` Ed Swierk
@ 2015-11-26 7:49 ` Jan Beulich
2015-11-27 2:11 ` Chao Peng
0 siblings, 1 reply; 22+ messages in thread
From: Jan Beulich @ 2015-11-26 7:49 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, xen-devel, Chao Peng
>>> On 26.11.15 at 00:27, <eswierk@skyportsystems.com> wrote:
> A few more data points: I also tested Xen 4.6 on VMware ESXi 5.5, and
> it yields similar results. Not surprising, since Fusion uses basically
> the same virtualization engine.
>
> However, ESXi offers many more choices of number of processors, number
> of cores, hyperthreading, etc. The weird processor ID assignment (0,
> 2, 4, 6, ...) occurs only with 4 or 8 processors, 1 core per socket,
> and no hyperthreading. If I change any of these parameters, the
> processor IDs become sequential.
>
> It appears in the 4- and 8-processor cases, VMware is emulating
> something like a Xeon E7340:
> https://github.com/deater/test_proc/blob/master/x86_64/x86_64.intel.6.15.11.
> xeon_e7340
>
> In fact someone asked a question about running Xen on this platform
> way back when:
> http://lists.xenproject.org/archives/html/xen-users/2008-05/msg00691.html
>
> Others of similar vintage assign processor IDs 0 and 3 on a
> 2-processor system:
> https://www.centos.org/forums/viewtopic.php?t=30255
>
> or even 0 and 6:
> http://serverfault.com/questions/302429/interpreting-cpuinfo
>
> So there are real hardware platforms with non-sequential processor
> IDs. They are quite ancient and don't support CAT, but that doesn't
> rule out the possibility of a newer or future platform behaving
> similarly.
Not supporting CAT is not a criteria, since the socket data setup
happens unconditionally. However (and as said before), non-
sequential processor IDs are fine. Non-sequential socket IDs are
what is problematic.
Jan
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-26 7:49 ` Jan Beulich
@ 2015-11-27 2:11 ` Chao Peng
2015-12-02 2:55 ` Nakajima, Jun
0 siblings, 1 reply; 22+ messages in thread
From: Chao Peng @ 2015-11-27 2:11 UTC (permalink / raw)
To: Jan Beulich, Ed Swierk; +Cc: Andrew Cooper, xen-devel
On Thu, Nov 26, 2015 at 12:49:42AM -0700, Jan Beulich wrote:
> >>> On 26.11.15 at 00:27, <eswierk@skyportsystems.com> wrote:
> > A few more data points: I also tested Xen 4.6 on VMware ESXi 5.5, and
> > it yields similar results. Not surprising, since Fusion uses basically
> > the same virtualization engine.
> >
> > However, ESXi offers many more choices of number of processors, number
> > of cores, hyperthreading, etc. The weird processor ID assignment (0,
> > 2, 4, 6, ...) occurs only with 4 or 8 processors, 1 core per socket,
> > and no hyperthreading. If I change any of these parameters, the
> > processor IDs become sequential.
> >
> > It appears in the 4- and 8-processor cases, VMware is emulating
> > something like a Xeon E7340:
> > https://github.com/deater/test_proc/blob/master/x86_64/x86_64.intel.6.15.11.
> > xeon_e7340
> >
> > In fact someone asked a question about running Xen on this platform
> > way back when:
> > http://lists.xenproject.org/archives/html/xen-users/2008-05/msg00691.html
> >
> > Others of similar vintage assign processor IDs 0 and 3 on a
> > 2-processor system:
> > https://www.centos.org/forums/viewtopic.php?t=30255
> >
> > or even 0 and 6:
> > http://serverfault.com/questions/302429/interpreting-cpuinfo
> >
> > So there are real hardware platforms with non-sequential processor
> > IDs. They are quite ancient and don't support CAT, but that doesn't
> > rule out the possibility of a newer or future platform behaving
> > similarly.
>
> Not supporting CAT is not a criteria, since the socket data setup
> happens unconditionally. However (and as said before), non-
> sequential processor IDs are fine. Non-sequential socket IDs are
> what is problematic.
I asked non-sequential socket ID problem internally but I don't know if
I can get a clear answer in the end, please just stay tuned for a while.
Thanks,
Chao
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-27 2:11 ` Chao Peng
@ 2015-12-02 2:55 ` Nakajima, Jun
0 siblings, 0 replies; 22+ messages in thread
From: Nakajima, Jun @ 2015-12-02 2:55 UTC (permalink / raw)
To: Chao Peng; +Cc: Andrew Cooper, Ed Swierk, Jan Beulich, xen-devel
BTW, we use "package ID", rather than "socket ID" in the SDM.
Assignment of the package IDs on a system is a BIOS matter. Basically
BIOS needs to assign package IDs to resolve APIC ID collision at early
boot time, and the convention is up to the vendor/or the specific
system configuration agents. And "contiguous package IDs" are not
required there to make it flexible. In fact, if you look at "Example
8-22. Compute the Number of Packages, Cores, and Processor
Relationships in a MP System", the sample code doesn't assume that
PACKAGE_IDs be continuous.
On Thu, Nov 26, 2015 at 6:11 PM, Chao Peng <chao.p.peng@linux.intel.com> wrote:
> On Thu, Nov 26, 2015 at 12:49:42AM -0700, Jan Beulich wrote:
>> >>> On 26.11.15 at 00:27, <eswierk@skyportsystems.com> wrote:
>> > A few more data points: I also tested Xen 4.6 on VMware ESXi 5.5, and
>> > it yields similar results. Not surprising, since Fusion uses basically
>> > the same virtualization engine.
>> >
>> > However, ESXi offers many more choices of number of processors, number
>> > of cores, hyperthreading, etc. The weird processor ID assignment (0,
>> > 2, 4, 6, ...) occurs only with 4 or 8 processors, 1 core per socket,
>> > and no hyperthreading. If I change any of these parameters, the
>> > processor IDs become sequential.
>> >
>> > It appears in the 4- and 8-processor cases, VMware is emulating
>> > something like a Xeon E7340:
>> > https://github.com/deater/test_proc/blob/master/x86_64/x86_64.intel.6.15.11.
>> > xeon_e7340
>> >
>> > In fact someone asked a question about running Xen on this platform
>> > way back when:
>> > http://lists.xenproject.org/archives/html/xen-users/2008-05/msg00691.html
>> >
>> > Others of similar vintage assign processor IDs 0 and 3 on a
>> > 2-processor system:
>> > https://www.centos.org/forums/viewtopic.php?t=30255
>> >
>> > or even 0 and 6:
>> > http://serverfault.com/questions/302429/interpreting-cpuinfo
>> >
>> > So there are real hardware platforms with non-sequential processor
>> > IDs. They are quite ancient and don't support CAT, but that doesn't
>> > rule out the possibility of a newer or future platform behaving
>> > similarly.
>>
>> Not supporting CAT is not a criteria, since the socket data setup
>> happens unconditionally. However (and as said before), non-
>> sequential processor IDs are fine. Non-sequential socket IDs are
>> what is problematic.
>
> I asked non-sequential socket ID problem internally but I don't know if
> I can get a clear answer in the end, please just stay tuned for a while.
>
> Thanks,
> Chao
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
--
Jun
Intel Open Source Technology Center
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-11-24 20:28 ` Ed Swierk
2015-11-25 9:11 ` Jan Beulich
@ 2015-12-02 13:48 ` Jan Beulich
2015-12-02 15:39 ` Ed Swierk
1 sibling, 1 reply; 22+ messages in thread
From: Jan Beulich @ 2015-12-02 13:48 UTC (permalink / raw)
To: Ed Swierk; +Cc: Andrew Cooper, xen-devel, Chao Peng
>>> On 24.11.15 at 21:28, <eswierk@skyportsystems.com> wrote:
> RFC. Boot tested on VMware Fusion, and on a 2-socket Xeon server.
Mind trying this one instead?
Jan
--- unstable.orig/xen/arch/x86/mpparse.c
+++ unstable/xen/arch/x86/mpparse.c
@@ -89,19 +89,14 @@ void __init set_nr_cpu_ids(unsigned int
void __init set_nr_sockets(void)
{
- /*
- * Count the actual cpus in the socket 0 and use it to calculate nr_sockets
- * so that the latter will be always >= the actual socket number in the
- * system even when APIC IDs from MP table are too sparse.
- */
- unsigned int cpus = bitmap_weight(phys_cpu_present_map.mask,
- boot_cpu_data.x86_max_cores *
- boot_cpu_data.x86_num_siblings);
-
- if ( cpus == 0 )
- cpus = 1;
-
- nr_sockets = DIV_ROUND_UP(num_processors + disabled_cpus, cpus);
+ nr_sockets = last_physid(phys_cpu_present_map)
+ / boot_cpu_data.x86_max_cores
+ / boot_cpu_data.x86_num_siblings + 1;
+ if (disabled_cpus)
+ nr_sockets += (disabled_cpus - 1)
+ / boot_cpu_data.x86_max_cores
+ / boot_cpu_data.x86_num_siblings + 1;
+ printk(XENLOG_DEBUG "nr_sockets: %u\n", nr_sockets);
}
/*
--- unstable.orig/xen/include/asm-x86/mpspec.h
+++ unstable/xen/include/asm-x86/mpspec.h
@@ -43,6 +43,19 @@ typedef struct physid_mask physid_mask_t
#define physid_isset(physid, map) test_bit(physid, (map).mask)
#define physid_test_and_set(physid, map) test_and_set_bit(physid, (map).mask)
+#define first_physid(map) find_first_bit((map).mask, \
+ MAX_APICS)
+#define next_physid(id, map) find_next_bit((map).mask, \
+ MAX_APICS, (id) + 1)
+#define last_physid(map) ({ \
+ const unsigned long *mask = (map).mask; \
+ unsigned int id, last = MAX_APICS; \
+ for (id = find_first_bit(mask, MAX_APICS); id < MAX_APICS; \
+ id = find_next_bit(mask, MAX_APICS, (id) + 1)) \
+ last = id; \
+ last; \
+})
+
#define physids_and(dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_or(dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
#define physids_clear(map) bitmap_zero((map).mask, MAX_APICS)
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion
2015-12-02 13:48 ` Jan Beulich
@ 2015-12-02 15:39 ` Ed Swierk
0 siblings, 0 replies; 22+ messages in thread
From: Ed Swierk @ 2015-12-02 15:39 UTC (permalink / raw)
To: Jan Beulich; +Cc: Andrew Cooper, xen-devel, Chao Peng
I tested on VMware Fusion with 3, 4 and 8 CPUs, and it works in all cases.
(XEN) Xen version 4.6.1-pre ( 4.6.1~pre-1skyport1)
(eswierk@skyportsystems.com) (gcc (Debian 5.2.1-19.1skyport1) 5.2.1
20150930) debug=n Wed Dec 2 07:22:20 PST 2015
(XEN) Bootloader: SYSLINUX 4.05 20140113
(XEN) Command line: xen console=com1,vga com1=115200 no-bootscrub
dom0_mem=2048M,max:2048M loglvl=all cpuinfo=1
(XEN) Video information:
(XEN) VGA is text mode 80x25, font 8x16
(XEN) Disc information:
(XEN) Found 1 MBR signatures
(XEN) Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN) 0000000000000000 - 000000000009f800 (usable)
(XEN) 000000000009f800 - 00000000000a0000 (reserved)
(XEN) 00000000000dc000 - 0000000000100000 (reserved)
(XEN) 0000000000100000 - 00000000bfef0000 (usable)
(XEN) 00000000bfef0000 - 00000000bfeff000 (ACPI data)
(XEN) 00000000bfeff000 - 00000000bff00000 (ACPI NVS)
(XEN) 00000000bff00000 - 00000000c0000000 (usable)
(XEN) 00000000f0000000 - 00000000f8000000 (reserved)
(XEN) 00000000fec00000 - 00000000fec10000 (reserved)
(XEN) 00000000fee00000 - 00000000fee01000 (reserved)
(XEN) 00000000fffe0000 - 0000000100000000 (reserved)
(XEN) 0000000100000000 - 00000001c0000000 (usable)
(XEN) ACPI: RSDP 000F6A10, 0024 (r2 PTLTD )
(XEN) ACPI: XSDT BFEF030B, 0054 (r1 INTEL 440BX 6040000 VMW 1324272)
(XEN) ACPI: FACP BFEFEE73, 00F4 (r4 INTEL 440BX 6040000 PTL F4240)
(XEN) ACPI: DSDT BFEF05B1, E8C2 (r1 PTLTD Custom 6040000 MSFT 3000001)
(XEN) ACPI: FACS BFEFFFC0, 0040
(XEN) ACPI: BOOT BFEF0589, 0028 (r1 PTLTD $SBFTBL$ 6040000 LTP 1)
(XEN) ACPI: APIC BFEF050F, 007A (r1 PTLTD APIC 6040000 LTP 0)
(XEN) ACPI: MCFG BFEF04D3, 003C (r1 PTLTD $PCITBL$ 6040000 LTP 1)
(XEN) ACPI: SRAT BFEF03C3, 0110 (r2 VMWARE MEMPLUG 6040000 VMW 1)
(XEN) ACPI: WAET BFEF039B, 0028 (r1 VMWARE VMW WAET 6040000 VMW 1)
(XEN) System RAM: 6143MB (6291004kB)
(XEN) SRAT: PXM 0 -> APIC 00 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 02 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 04 -> Node 0
(XEN) SRAT: PXM 0 -> APIC 06 -> Node 0
(XEN) SRAT: Node 0 PXM 0 0-a0000
(XEN) SRAT: Node 0 PXM 0 100000-10000000
(XEN) SRAT: Node 0 PXM 0 10000000-c0000000
(XEN) SRAT: Node 0 PXM 0 100000000-1c0000000
(XEN) NUMA: Allocated memnodemap from 1bdc50000 - 1bdc52000
(XEN) NUMA: Using 8 for the hash shift.
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000f6a80
(XEN) DMI present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x1008
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:1004,1:0], pm1x_evt[1:1000,1:0]
(XEN) ACPI: wakeup_vec[bfefffcc], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee00000
(XEN) ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
(XEN) Processor #0 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x02] enabled)
(XEN) Processor #2 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x04] enabled)
(XEN) Processor #4 6:6 APIC version 21
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x06] enabled)
(XEN) Processor #6 6:6 APIC version 21
(XEN) ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
(XEN) ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
(XEN) ACPI: IOAPIC (id[0x01] address[0xfec00000] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 1, version 17, address 0xfec00000, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) Enabling APIC mode: Flat. Using 1 I/O APICs
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 4 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 760 MSI/MSI-X
(XEN) Not enabling x2APIC: depends on iommu_supports_eim.
(XEN) CPU: Physical Processor ID: 0
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) xstate_init: using cntxt_size: 0x340 and states: 0x7
(XEN) CPU0: No MCE banks present. Machine check support disabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Initializing CPU#0
(XEN) Detected 2592.669 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table ffff82d0802bd090 -> ffff82d0802be2c0
(XEN) PCI: MCFG configuration 0: base f0000000 segment 0000 buses 00 - 7f
(XEN) PCI: MCFG area at f0000000 reserved in E820
(XEN) PCI: Using MCFG for segment 0000 bus 00-7f
(XEN) I/O virtualisation disabled
(XEN) CPU0: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) nr_sockets: 7
(XEN) ENABLING IO-APIC IRQs
(XEN) -> Using new ACK method
(XEN) ..TIMER: vector=0xF0 apic1=0 pin1=2 apic2=-1 pin2=-1
(XEN) Platform timer is 3.579MHz ACPI PM Timer
(XEN) Allocated console ring of 32 KiB.
(XEN) mwait-idle: MWAIT substates: 0x10
(XEN) mwait-idle: v0.4 model 0x46
(XEN) mwait-idle: lapic_timer_reliable_states 0xffffffff
(XEN) VMX: Supported advanced features:
(XEN) - APIC TPR shadow
(XEN) - Extended Page Tables (EPT)
(XEN) - Virtual-Processor Identifiers (VPID)
(XEN) - Virtual NMI
(XEN) - MSR direct-access bitmap
(XEN) - Unrestricted Guest
(XEN) HVM: ASIDs enabled.
(XEN) HVM: VMX enabled
(XEN) HVM: Hardware Assisted Paging (HAP) not detected
(XEN) HVM: PVH mode not supported on this platform
(XEN) CPU 0 APIC 0 -> Node 0
(XEN) CPU 1 APIC 2 -> Node 0
(XEN) Booting processor 1/2 eip 8f000
(XEN) Initializing CPU#1
(XEN) CPU: Physical Processor ID: 2
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU1: No MCE banks present. Machine check support disabled
(XEN) CPU1: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 2 APIC 4 -> Node 0
(XEN) Booting processor 2/4 eip 8f000
(XEN) Initializing CPU#2
(XEN) CPU: Physical Processor ID: 4
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU2: No MCE banks present. Machine check support disabled
(XEN) CPU2: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) CPU 3 APIC 6 -> Node 0
(XEN) Booting processor 3/6 eip 8f000
(XEN) Initializing CPU#3
(XEN) CPU: Physical Processor ID: 6
(XEN) CPU: L1 I cache: 32K, L1 D cache: 32K
(XEN) CPU: L2 cache: 256K
(XEN) CPU: L3 cache: 6144K
(XEN) CPU3: No MCE banks present. Machine check support disabled
(XEN) CPU3: Intel(R) Core(TM) i7-4960HQ CPU @ 2.60GHz stepping 01
(XEN) Brought up 4 CPUs
(XEN) ACPI sleep modes: S3
(XEN) VPMU: disabled
(XEN) Dom0 has maximum 600 PIRQs
(XEN) NX (Execute Disable) protection active
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Xen kernel: 64-bit, lsb, compat32
(XEN) Dom0 kernel: 64-bit, PAE, lsb, paddr 0x1000000 -> 0x2e00000
(XEN) PHYSICAL MEMORY ARRANGEMENT:
(XEN) Dom0 alloc.: 00000001b4000000->00000001b8000000 (504788 pages
to be allocated)
(XEN) Init. ramdisk: 00000001bf3d4000->00000001bffffc00
(XEN) VIRTUAL MEMORY ARRANGEMENT:
(XEN) Loaded kernel: ffffffff81000000->ffffffff82e00000
(XEN) Init. ramdisk: ffffffff82e00000->ffffffff83a2bc00
(XEN) Phys-Mach map: ffffffff83a2c000->ffffffff83e2c000
(XEN) Start info: ffffffff83e2c000->ffffffff83e2c4b4
(XEN) Page tables: ffffffff83e2d000->ffffffff83e50000
(XEN) Boot stack: ffffffff83e50000->ffffffff83e51000
(XEN) TOTAL: ffffffff80000000->ffffffff84000000
(XEN) ENTRY ADDRESS: ffffffff81e1e1f0
(XEN) Dom0 has maximum 4 VCPUs
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: Nothing (Rate-limited: Errors and warnings)
(XEN) Xen is relinquishing VGA console.
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch
input to Xen)
(XEN) Freed 288kB init memory.
mapping kernel into physical memory
about to get started...
--Ed
On Wed, Dec 2, 2015 at 5:48 AM, Jan Beulich <JBeulich@suse.com> wrote:
>>>> On 24.11.15 at 21:28, <eswierk@skyportsystems.com> wrote:
>> RFC. Boot tested on VMware Fusion, and on a 2-socket Xeon server.
>
> Mind trying this one instead?
>
> Jan
>
> --- unstable.orig/xen/arch/x86/mpparse.c
> +++ unstable/xen/arch/x86/mpparse.c
> @@ -89,19 +89,14 @@ void __init set_nr_cpu_ids(unsigned int
>
> void __init set_nr_sockets(void)
> {
> - /*
> - * Count the actual cpus in the socket 0 and use it to calculate nr_sockets
> - * so that the latter will be always >= the actual socket number in the
> - * system even when APIC IDs from MP table are too sparse.
> - */
> - unsigned int cpus = bitmap_weight(phys_cpu_present_map.mask,
> - boot_cpu_data.x86_max_cores *
> - boot_cpu_data.x86_num_siblings);
> -
> - if ( cpus == 0 )
> - cpus = 1;
> -
> - nr_sockets = DIV_ROUND_UP(num_processors + disabled_cpus, cpus);
> + nr_sockets = last_physid(phys_cpu_present_map)
> + / boot_cpu_data.x86_max_cores
> + / boot_cpu_data.x86_num_siblings + 1;
> + if (disabled_cpus)
> + nr_sockets += (disabled_cpus - 1)
> + / boot_cpu_data.x86_max_cores
> + / boot_cpu_data.x86_num_siblings + 1;
> + printk(XENLOG_DEBUG "nr_sockets: %u\n", nr_sockets);
> }
>
> /*
> --- unstable.orig/xen/include/asm-x86/mpspec.h
> +++ unstable/xen/include/asm-x86/mpspec.h
> @@ -43,6 +43,19 @@ typedef struct physid_mask physid_mask_t
> #define physid_isset(physid, map) test_bit(physid, (map).mask)
> #define physid_test_and_set(physid, map) test_and_set_bit(physid, (map).mask)
>
> +#define first_physid(map) find_first_bit((map).mask, \
> + MAX_APICS)
> +#define next_physid(id, map) find_next_bit((map).mask, \
> + MAX_APICS, (id) + 1)
> +#define last_physid(map) ({ \
> + const unsigned long *mask = (map).mask; \
> + unsigned int id, last = MAX_APICS; \
> + for (id = find_first_bit(mask, MAX_APICS); id < MAX_APICS; \
> + id = find_next_bit(mask, MAX_APICS, (id) + 1)) \
> + last = id; \
> + last; \
> +})
> +
> #define physids_and(dst, src1, src2) bitmap_and((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
> #define physids_or(dst, src1, src2) bitmap_or((dst).mask, (src1).mask, (src2).mask, MAX_APICS)
> #define physids_clear(map) bitmap_zero((map).mask, MAX_APICS)
>
>
^ permalink raw reply [flat|nested] 22+ messages in thread
end of thread, other threads:[~2015-12-02 15:39 UTC | newest]
Thread overview: 22+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-11-20 1:22 Crash in set_cpu_sibling_map() booting Xen 4.6.0 on Fusion Ed Swierk
2015-11-20 10:25 ` Jan Beulich
2015-11-20 10:37 ` Andrew Cooper
2015-11-20 18:53 ` Ed Swierk
2015-11-21 1:21 ` Ed Swierk
2015-11-23 1:10 ` Chao Peng
2015-11-23 5:39 ` Chao Peng
2015-11-23 10:18 ` Jan Beulich
2015-11-23 16:36 ` Ed Swierk
2015-11-24 10:34 ` Jan Beulich
2015-11-24 14:13 ` Ed Swierk
2015-11-24 15:20 ` Jan Beulich
2015-11-24 20:28 ` Ed Swierk
2015-11-25 9:11 ` Jan Beulich
2015-12-02 13:48 ` Jan Beulich
2015-12-02 15:39 ` Ed Swierk
2015-11-25 7:48 ` Chao Peng
2015-11-25 9:04 ` Jan Beulich
2015-11-25 23:27 ` Ed Swierk
2015-11-26 7:49 ` Jan Beulich
2015-11-27 2:11 ` Chao Peng
2015-12-02 2:55 ` Nakajima, Jun
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.