* Crash report: Broken NUMA distance map causes crash on arm64 system
@ 2018-10-23 10:30 John Garry
2018-10-25 11:01 ` John Garry
2018-10-30 9:26 ` Peter Zijlstra
0 siblings, 2 replies; 13+ messages in thread
From: John Garry @ 2018-10-23 10:30 UTC (permalink / raw)
To: linux-arm-kernel, linux-kernel, Will Deacon, Catalin Marinas,
Rob Herring, devicetree, Frank Rowand, Ingo Molnar,
Peter Zijlstra
Cc: Linuxarm
Hi all,
I have stumbled upon this crash on my arm64 system:
[ 7.040874] SMP: Total of 64 processors activated.
[ 7.045720] CPU features: detected: GIC system register CPU interface
[ 7.052240] CPU features: detected: 32-bit EL0 Support
[ 7.144026] CPU: All CPU(s) started at EL2
[ 7.148298] alternatives: patching kernel code
[ 7.155277] Unable to handle kernel NULL pointer dereference at
virtual address 0000000000000000
[ 7.164163] Mem abort info:
[ 7.166978] ESR = 0x96000004
[ 7.170061] Exception class = DABT (current EL), IL = 32 bits
[ 7.176043] SET = 0, FnV = 0
[ 7.179121] EA = 0, S1PTW = 0
[ 7.182291] Data abort info:
[ 7.185193] ISV = 0, ISS = 0x00000004
[ 7.189066] CM = 0, WnR = 0
[ 7.192056] [0000000000000000] user address but active_mm is swapper
[ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 7.204107] Modules linked in:
[ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
4.19.0-00002-g3ed52fd-dirty #807
[ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
[ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
[ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
[ 7.237797] sp : ffff00000947bc60
[ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
[ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
[ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
[ 7.257235] x23: 0000000000000010 x22: 0000000000000001
[ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
[ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
[ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
[ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
[ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
[ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
[ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
[ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
[ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
[ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
[ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
[ 7.321613] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
[ 7.328382] Call trace:
[ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
[ 7.335420] destroy_sched_domain+0x20/0x70
[ 7.339642] cpu_attach_domain+0xc8/0x2e8
[ 7.343687] build_sched_domains+0xd44/0xde0
[ 7.347996] sched_init_domains+0x68/0x88
[ 7.352044] sched_init_smp+0x2c/0x7c
[ 7.355738] kernel_init_freeable+0xdc/0x244
[ 7.360049] kernel_init+0x10/0x108
[ 7.363567] ret_from_fork+0x10/0x18
[ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
[ 7.373400] ---[ end trace 8150af869a14363f ]---
[ 7.378076] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 7.378076]
[ 7.387320] SMP: stopping secondary CPUs
[ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b
[ 7.391297] ]---
I will straightaway note that I have been fiddling with my board's
device tree, specifically (breaking!) the NUMA distance map, like this:
distance-matrix = <0 0 10>,
<0 1 15>,
<0 2 20>,
<0 3 25>,
<1 0 15>,
<1 1 10>,
<1 2 25>,
<1 3 30>,
<2 0 20>,
<2 1 25>,
<2 2 10>,
<2 3 15>,
<3 0 10>,* should be same as 0->3 and > 10
<3 1 10>,* should be same as 1->3 and > 10
<3 2 15>,
<3 3 15>;* should be 10
However I don't think that this should crash the kernel. I'd say
of_numa_parse_distance_map_v1() should robustly handle broken maps,
probably by erroring and causing NUMA to be disabled.
However I am interested to know if the scheduler crash is a real problem.
The kernel is 4.19. Full log is below.
Thanks,
John
Full log:
EFI stub: Booting Linux Kernel...
SmiGraphicsOutputQueryMode +
SmiGraphicsOutputQueryMode -
SmiGraphicsOutputQueryMode +
SmiGraphicsOutputQueryMode -
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
[SAL_ClearAffiliationSMP,325]it is going to hard reset
dev:0x500E004AAAAAAA1F
SAS ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
OHCI ExitBootServicesEvent
IPMI ExitBootService Event
TransferSmbiosToBMC EVENT.
Transfer Smbios Table To iBMC Success.
GetVariable Status : Not Found.
[ 0.000000] Booting Linux on physical CPU 0x0000010000 [0x410fd082]
[ 0.000000] Linux version 4.19.0-00002-g3ed52fd-dirty
(johnpgarry@johnpgarry-ThinkCentre-M93p) (gcc version 7.3.1 20180425
[linaro-7.3-2018.05-rc1 revision
38aec9a676236eaa42ca03ccb3a6c1dd0182c29f] (Linaro GCC 7.3-2018.05-rc1))
#807 SMP PREEMPT Tue Oct 23 11:02:19 BST 2018
[ 0.000000] Machine model: Hisilicon Hip07 D05 Development Board
[ 0.000000] earlycon: pl11 at MMIO 0x00000000602b0000 (options '')
[ 0.000000] bootconsole [pl11] enabled
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: EFI v2.60 by EDK II
[ 0.000000] efi: SMBIOS=0x3f050000 SMBIOS 3.0=0x39b00000
ACPI=0x39bd0000 ACPI 2.0=0x39bd0014 MEMATTR=0x3cc99018
[ 0.000000] crashkernel reserved: 0x0000000002000000 -
0x0000000012000000 (256 MB)
[ 0.000000] cma: Reserved 32 MiB at 0x000000003d000000
[ 0.000000] earlycon: pl11 at MMIO32 0x00000000602b0000 (options
'115200n8')
[ 0.000000] ------------[ cut here ]------------
[ 0.000000] console 'pl11' already registered
[ 0.000000] WARNING: CPU: 0 PID: 0 at kernel/printk/printk.c:2586
register_console+0x58/0x3c8
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
4.19.0-00002-g3ed52fd-dirty #807
[ 0.000000] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 0.000000] pstate: 60000085 (nZCv daIf -PAN -UAO)
[ 0.000000] pc : register_console+0x58/0x3c8
[ 0.000000] lr : register_console+0x58/0x3c8
[ 0.000000] sp : ffff000009103de0
[ 0.000000] x29: ffff000009103de0 x28: 0000000000fb0018
[ 0.000000] x27: 000000003cc81018 x26: ffff0000090d1098
[ 0.000000] x25: 0000000000004b10 x24: ffff0000090e4348
[ 0.000000] x23: ffff0000090d1098 x22: ffff000009262000
[ 0.000000] x21: ffff0000091fe150 x20: 0000000000004b10
[ 0.000000] x19: ffff0000091fe150 x18: ffffffffffffffff
[ 0.000000] x17: 0000000000000000 x16: 0000000000000000
[ 0.000000] x15: ffff0000091096c8 x14: ffff0000892626bf
[ 0.000000] x13: ffff0000092626cd x12: ffff000009109940
[ 0.000000] x11: ffff0000085ecd98 x10: ffff000009103ab0
[ 0.000000] x9 : 000000000000000d x8 : 7369676572207964
[ 0.000000] x7 : 6165726c61202731 x6 : 000000000000000d
[ 0.000000] x5 : 0000000000000000 x4 : 0000000000000000
[ 0.000000] x3 : ffffffffffffffff x2 : ffff0000091210f8
[ 0.000000] x1 : 0000000000000000 x0 : 0000000000000000
[ 0.000000] Call trace:
[ 0.000000] register_console+0x58/0x3c8
[ 0.000000] of_setup_earlycon+0x20c/0x22c
[ 0.000000] early_init_dt_scan_chosen_stdout+0x158/0x19c
[ 0.000000] acpi_boot_table_init+0x150/0x184
[ 0.000000] setup_arch+0x1cc/0x530
[ 0.000000] start_kernel+0x80/0x418
[ 0.000000] random: get_random_bytes called from
print_oops_end_marker+0x48/0x68 with crng_init=0
[ 0.000000] ---[ end trace 0000000000000000 ]---
[ 0.000000] NUMA: NODE_DATA [mem 0x1ffbffe840-0x1ffbffffff]
[ 0.000000] NUMA: Initmem setup node 1 [<memory-less node>]
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf41fc0-0x41ffbf4377f]
[ 0.000000] NUMA: NODE_DATA(1) on node 2
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf40800-0x41ffbf41fbf]
[ 0.000000] NUMA: Initmem setup node 3 [<memory-less node>]
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf3f040-0x41ffbf407ff]
[ 0.000000] NUMA: NODE_DATA(3) on node 2
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000000000000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x0000041ffbffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x0000000031902fff]
[ 0.000000] node 0: [mem 0x0000000031903000-0x0000000031906fff]
[ 0.000000] node 0: [mem 0x0000000031907000-0x000000003999ffff]
[ 0.000000] node 0: [mem 0x00000000399a0000-0x0000000039a3ffff]
[ 0.000000] node 0: [mem 0x0000000039a40000-0x0000000039a8ffff]
[ 0.000000] node 0: [mem 0x0000000039a90000-0x0000000039a9ffff]
[ 0.000000] node 0: [mem 0x0000000039aa0000-0x0000000039aeffff]
[ 0.000000] node 0: [mem 0x0000000039af0000-0x0000000039bbffff]
[ 0.000000] node 0: [mem 0x0000000039bc0000-0x0000000039bdffff]
[ 0.000000] node 0: [mem 0x0000000039be0000-0x000000003a05ffff]
[ 0.000000] node 0: [mem 0x000000003a060000-0x000000003f04ffff]
[ 0.000000] node 0: [mem 0x000000003f050000-0x000000003f07ffff]
[ 0.000000] node 0: [mem 0x000000003f080000-0x000000003fbfffff]
[ 0.000000] node 0: [mem 0x0000001040000000-0x0000001ffbffffff]
[ 0.000000] node 2: [mem 0x0000041000000000-0x0000041ffbffffff]
[ 0.000000] Initmem setup node 0 [mem
0x0000000000000000-0x0000001ffbffffff]
[ 0.000000] Could not find start_pfn for node 1
[ 0.000000] Initmem setup node 1 [mem
0x0000000000000000-0x0000000000000000]
[ 0.000000] Initmem setup node 2 [mem
0x0000041000000000-0x0000041ffbffffff]
[ 0.000000] Could not find start_pfn for node 3
[ 0.000000] Initmem setup node 3 [mem
0x0000000000000000-0x0000000000000000]
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] percpu: Embedded 23 pages/cpu @(____ptrval____) s56408
r8192 d29608 u94208
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for EL2 vector hardening
[ 0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[ 0.000000] Built 4 zonelists, mobility grouping on. Total pages:
32996880
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: BOOT_IMAGE=/john/Image rdinit=/init
crashkernel=256M@32M console=ttyAMA0,115200
earlycon=pl011,mmio,0x602b0000 acpi=off pcie_aspm=off earlycon
[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[ 0.000000] log_buf_len total cpu_extra contributions: 258048 bytes
[ 0.000000] log_buf_len min size: 131072 bytes
[ 0.000000] log_buf_len: 524288 bytes
[ 0.000000] early log buf free: 123488(94%)
[ 0.000000] software IO TLB: mapped [mem 0x359a0000-0x399a0000] (64MB)
[ 0.000000] Memory: 131332260K/134082560K available (10620K kernel
code, 1370K rwdata, 4888K rodata, 1344K init, 380K bss, 2717532K
reserved, 32768K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=64, Nodes=4
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv3: Distributor has no Range Selector support
[ 0.000000] GICv3: VLPI support, direct LPI support
[ 0.000000] ITS [mem 0x4c000000-0x4c03ffff]
[ 0.000000] ITS@0x000000004c000000: Using ITS number 0
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000000004c000000: allocated 524288 Devices
@1fbb000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000000004c000000: allocated 65536 Virtual CPUs
@1fbb980000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000000004c000000: allocated 512 Interrupt
Collections @1fbb929000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x6c000000-0x6c03ffff]
[ 0.000000] ITS@0x000000006c000000: Using ITS number 1
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000000006c000000: allocated 524288 Devices
@1fbac00000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000000006c000000: allocated 65536 Virtual CPUs
@1fbba00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000000006c000000: allocated 512 Interrupt
Collections @1fbb92a000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0xc6000000-0xc603ffff]
[ 0.000000] ITS@0x00000000c6000000: Using ITS number 2
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000000c6000000: allocated 524288 Devices
@1fba800000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000000c6000000: allocated 65536 Virtual CPUs
@1fbba80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt
Collections @1fbb92c000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x8c6000000-0x8c603ffff]
[ 0.000000] ITS@0x00000008c6000000: Using ITS number 3
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000008c6000000: allocated 524288 Devices
@1fba400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000008c6000000: allocated 65536 Virtual CPUs
@1fbbb00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000008c6000000: allocated 512 Interrupt
Collections @1fbb92e000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x4004c000000-0x4004c03ffff]
[ 0.000000] ITS@0x000004004c000000: Using ITS number 4
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000004004c000000: allocated 524288 Devices
@1fba000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000004004c000000: allocated 65536 Virtual CPUs
@1fbbb80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000004004c000000: allocated 512 Interrupt
Collections @1fbb92f000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x4006c000000-0x4006c03ffff]
[ 0.000000] ITS@0x000004006c000000: Using ITS number 5
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000004006c000000: allocated 524288 Devices
@1fb9800000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000004006c000000: allocated 65536 Virtual CPUs
@1fb9c80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000004006c000000: allocated 512 Interrupt
Collections @1fb9c10000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x400c6000000-0x400c603ffff]
[ 0.000000] ITS@0x00000400c6000000: Using ITS number 6
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000400c6000000: allocated 524288 Devices
@1fb9400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000400c6000000: allocated 65536 Virtual CPUs
@1fb9d00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000400c6000000: allocated 512 Interrupt
Collections @1fb9c11000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x408c6000000-0x408c603ffff]
[ 0.000000] ITS@0x00000408c6000000: Using ITS number 7
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000408c6000000: allocated 524288 Devices
@1fb9000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000408c6000000: allocated 65536 Virtual CPUs
@1fb9d80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000408c6000000: allocated 512 Interrupt
Collections @1fb9c13000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] GIC: using LPI property table @0x0000001fb9c40000
[ 0.000000] ITS: Using DirectLPI for VPE invalidation
[ 0.000000] ITS: Enabling GICv4 support
[ 0.000000] GICv3: CPU0: found redistributor 10000 region
0:0x000000004d100000
[ 0.000000] CPU0: using LPI pending table @0x0000001fb9c50000
[ 0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff
max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[ 0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps
every 4398046511100ns
[ 0.008248] Console: colour dummy device 80x25
[ 0.012791] mempolicy: Enabling automatic NUMA balancing. Configure
with numa_balancing= or the kernel.numa_balancing sysctl
[ 0.024159] Calibrating delay loop (skipped), value calculated using
timer frequency.. 100.00 BogoMIPS (lpj=200000)
[ 0.034713] pid_max: default: 65536 minimum: 512
[ 0.039438] Security Framework initialized
[ 0.065597] Dentry cache hash table entries: 8388608 (order: 14,
67108864 bytes)
[ 0.084013] Inode-cache hash table entries: 4194304 (order: 13,
33554432 bytes)
[ 0.091758] Mount-cache hash table entries: 131072 (order: 8, 1048576
bytes)
[ 0.099226] Mountpoint-cache hash table entries: 131072 (order: 8,
1048576 bytes)
[ 0.150842] ASID allocator initialised with 32768 entries
[ 0.164305] rcu: Hierarchical SRCU implementation.
[ 0.177184] Platform MSI: interrupt-controller@4c000000 domain created
[ 0.183784] Platform MSI: interrupt-controller@6c000000 domain created
[ 0.190384] Platform MSI: interrupt-controller@c6000000 domain created
[ 0.196983] Platform MSI: interrupt-controller@8,c6000000 domain created
[ 0.203758] Platform MSI: interrupt-controller@400,4c000000 domain
created
[ 0.210708] Platform MSI: interrupt-controller@400,6c000000 domain
created
[ 0.217659] Platform MSI: interrupt-controller@400,c6000000 domain
created
[ 0.224610] Platform MSI: interrupt-controller@408,c6000000 domain
created
[ 0.231606] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@4c000000 domain created
[ 0.240491] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@6c000000 domain created
[ 0.249375] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@c6000000 domain created
[ 0.258260] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@8,c6000000 domain
created
[ 0.267320] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,4c000000 domain
created
[ 0.276556] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,6c000000 domain
created
[ 0.285792] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,c6000000 domain
created
[ 0.295029] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@408,c6000000 domain
created
[ 0.304714] Remapping and enabling EFI services.
[ 0.317387] smp: Bringing up secondary CPUs ...
[ 0.404570] Detected PIPT I-cache on CPU1
[ 0.404582] GICv3: CPU1: found redistributor 10001 region
0:0x000000004d140000
[ 0.404611] CPU1: using LPI pending table @0x0000001fb5dc0000
[ 0.404728] CPU1: Booted secondary processor 0x0000010001 [0x410fd082]
[ 0.487192] Detected PIPT I-cache on CPU2
[ 0.487198] GICv3: CPU2: found redistributor 10002 region
0:0x000000004d180000
[ 0.487224] CPU2: using LPI pending table @0x0000001fb5e10000
[ 0.487329] CPU2: Booted secondary processor 0x0000010002 [0x410fd082]
[ 0.569818] Detected PIPT I-cache on CPU3
[ 0.569824] GICv3: CPU3: found redistributor 10003 region
0:0x000000004d1c0000
[ 0.569851] CPU3: using LPI pending table @0x0000001fb5e30000
[ 0.569953] CPU3: Booted secondary processor 0x0000010003 [0x410fd082]
[ 0.652445] Detected PIPT I-cache on CPU4
[ 0.652453] GICv3: CPU4: found redistributor 10100 region
0:0x000000004d200000
[ 0.652480] CPU4: using LPI pending table @0x0000001fb5e40000
[ 0.652584] CPU4: Booted secondary processor 0x0000010100 [0x410fd082]
[ 0.735070] Detected PIPT I-cache on CPU5
[ 0.735077] GICv3: CPU5: found redistributor 10101 region
0:0x000000004d240000
[ 0.735103] CPU5: using LPI pending table @0x0000001fb5e90000
[ 0.735206] CPU5: Booted secondary processor 0x0000010101 [0x410fd082]
[ 0.817697] Detected PIPT I-cache on CPU6
[ 0.817704] GICv3: CPU6: found redistributor 10102 region
0:0x000000004d280000
[ 0.817731] CPU6: using LPI pending table @0x0000001fb5ea0000
[ 0.817834] CPU6: Booted secondary processor 0x0000010102 [0x410fd082]
[ 0.900324] Detected PIPT I-cache on CPU7
[ 0.900331] GICv3: CPU7: found redistributor 10103 region
0:0x000000004d2c0000
[ 0.900358] CPU7: using LPI pending table @0x0000001fb5eb0000
[ 0.900461] CPU7: Booted secondary processor 0x0000010103 [0x410fd082]
[ 0.982952] Detected PIPT I-cache on CPU8
[ 0.982961] GICv3: CPU8: found redistributor 10200 region
0:0x000000004d300000
[ 0.982988] CPU8: using LPI pending table @0x0000001fb5f00000
[ 0.983092] CPU8: Booted secondary processor 0x0000010200 [0x410fd082]
[ 1.065578] Detected PIPT I-cache on CPU9
[ 1.065585] GICv3: CPU9: found redistributor 10201 region
0:0x000000004d340000
[ 1.065611] CPU9: using LPI pending table @0x0000001fb5f20000
[ 1.065714] CPU9: Booted secondary processor 0x0000010201 [0x410fd082]
[ 1.148205] Detected PIPT I-cache on CPU10
[ 1.148213] GICv3: CPU10: found redistributor 10202 region
0:0x000000004d380000
[ 1.148240] CPU10: using LPI pending table @0x0000001fb5f30000
[ 1.148343] CPU10: Booted secondary processor 0x0000010202 [0x410fd082]
[ 1.230833] Detected PIPT I-cache on CPU11
[ 1.230841] GICv3: CPU11: found redistributor 10203 region
0:0x000000004d3c0000
[ 1.230868] CPU11: using LPI pending table @0x0000001fb5f80000
[ 1.230971] CPU11: Booted secondary processor 0x0000010203 [0x410fd082]
[ 1.313460] Detected PIPT I-cache on CPU12
[ 1.313469] GICv3: CPU12: found redistributor 10300 region
0:0x000000004d400000
[ 1.313496] CPU12: using LPI pending table @0x0000001fb5fa0000
[ 1.313602] CPU12: Booted secondary processor 0x0000010300 [0x410fd082]
[ 1.396086] Detected PIPT I-cache on CPU13
[ 1.396094] GICv3: CPU13: found redistributor 10301 region
0:0x000000004d440000
[ 1.396121] CPU13: using LPI pending table @0x0000001fb5fb0000
[ 1.396226] CPU13: Booted secondary processor 0x0000010301 [0x410fd082]
[ 1.478713] Detected PIPT I-cache on CPU14
[ 1.478722] GICv3: CPU14: found redistributor 10302 region
0:0x000000004d480000
[ 1.478748] CPU14: using LPI pending table @0x0000001fb5fc0000
[ 1.478852] CPU14: Booted secondary processor 0x0000010302 [0x410fd082]
[ 1.561340] Detected PIPT I-cache on CPU15
[ 1.561348] GICv3: CPU15: found redistributor 10303 region
0:0x000000004d4c0000
[ 1.561374] CPU15: using LPI pending table @0x0000001fb5820000
[ 1.561477] CPU15: Booted secondary processor 0x0000010303 [0x410fd082]
[ 1.643629] Detected PIPT I-cache on CPU16
[ 1.643641] GICv3: CPU16: found redistributor 30000 region
1:0x000000006d100000
[ 1.643680] CPU16: using LPI pending table @0x0000001fb5860000
[ 1.643828] CPU16: Booted secondary processor 0x0000030000 [0x410fd082]
[ 1.726259] Detected PIPT I-cache on CPU17
[ 1.726268] GICv3: CPU17: found redistributor 30001 region
1:0x000000006d140000
[ 1.726305] CPU17: using LPI pending table @0x0000001fb58f0000
[ 1.726449] CPU17: Booted secondary processor 0x0000030001 [0x410fd082]
[ 1.808890] Detected PIPT I-cache on CPU18
[ 1.808900] GICv3: CPU18: found redistributor 30002 region
1:0x000000006d180000
[ 1.808936] CPU18: using LPI pending table @0x0000001fb5970000
[ 1.809080] CPU18: Booted secondary processor 0x0000030002 [0x410fd082]
[ 1.891523] Detected PIPT I-cache on CPU19
[ 1.891533] GICv3: CPU19: found redistributor 30003 region
1:0x000000006d1c0000
[ 1.891570] CPU19: using LPI pending table @0x0000001fb59b0000
[ 1.891713] CPU19: Booted secondary processor 0x0000030003 [0x410fd082]
[ 1.974158] Detected PIPT I-cache on CPU20
[ 1.974171] GICv3: CPU20: found redistributor 30100 region
1:0x000000006d200000
[ 1.974209] CPU20: using LPI pending table @0x0000001fb5a20000
[ 1.974356] CPU20: Booted secondary processor 0x0000030100 [0x410fd082]
[ 2.056788] Detected PIPT I-cache on CPU21
[ 2.056797] GICv3: CPU21: found redistributor 30101 region
1:0x000000006d240000
[ 2.056834] CPU21: using LPI pending table @0x0000001fb5ab0000
[ 2.056979] CPU21: Booted secondary processor 0x0000030101 [0x410fd082]
[ 2.139420] Detected PIPT I-cache on CPU22
[ 2.139430] GICv3: CPU22: found redistributor 30102 region
1:0x000000006d280000
[ 2.139466] CPU22: using LPI pending table @0x0000001fb5af0000
[ 2.139610] CPU22: Booted secondary processor 0x0000030102 [0x410fd082]
[ 2.222053] Detected PIPT I-cache on CPU23
[ 2.222063] GICv3: CPU23: found redistributor 30103 region
1:0x000000006d2c0000
[ 2.222100] CPU23: using LPI pending table @0x0000001fb5b70000
[ 2.222244] CPU23: Booted secondary processor 0x0000030103 [0x410fd082]
[ 2.304688] Detected PIPT I-cache on CPU24
[ 2.304701] GICv3: CPU24: found redistributor 30200 region
1:0x000000006d300000
[ 2.304739] CPU24: using LPI pending table @0x0000001fb5bb0000
[ 2.304888] CPU24: Booted secondary processor 0x0000030200 [0x410fd082]
[ 2.387319] Detected PIPT I-cache on CPU25
[ 2.387329] GICv3: CPU25: found redistributor 30201 region
1:0x000000006d340000
[ 2.387365] CPU25: using LPI pending table @0x0000001fb5430000
[ 2.387510] CPU25: Booted secondary processor 0x0000030201 [0x410fd082]
[ 2.469951] Detected PIPT I-cache on CPU26
[ 2.469961] GICv3: CPU26: found redistributor 30202 region
1:0x000000006d380000
[ 2.469998] CPU26: using LPI pending table @0x0000001fb54a0000
[ 2.470143] CPU26: Booted secondary processor 0x0000030202 [0x410fd082]
[ 2.552585] Detected PIPT I-cache on CPU27
[ 2.552595] GICv3: CPU27: found redistributor 30203 region
1:0x000000006d3c0000
[ 2.552632] CPU27: using LPI pending table @0x0000001fb54f0000
[ 2.552777] CPU27: Booted secondary processor 0x0000030203 [0x410fd082]
[ 2.635220] Detected PIPT I-cache on CPU28
[ 2.635234] GICv3: CPU28: found redistributor 30300 region
1:0x000000006d400000
[ 2.635271] CPU28: using LPI pending table @0x0000001fb5570000
[ 2.635419] CPU28: Booted secondary processor 0x0000030300 [0x410fd082]
[ 2.717851] Detected PIPT I-cache on CPU29
[ 2.717861] GICv3: CPU29: found redistributor 30301 region
1:0x000000006d440000
[ 2.717898] CPU29: using LPI pending table @0x0000001fb55b0000
[ 2.718043] CPU29: Booted secondary processor 0x0000030301 [0x410fd082]
[ 2.800486] Detected PIPT I-cache on CPU30
[ 2.800496] GICv3: CPU30: found redistributor 30302 region
1:0x000000006d480000
[ 2.800533] CPU30: using LPI pending table @0x0000001fb5790000
[ 2.800679] CPU30: Booted secondary processor 0x0000030302 [0x410fd082]
[ 2.883122] Detected PIPT I-cache on CPU31
[ 2.883133] GICv3: CPU31: found redistributor 30303 region
1:0x000000006d4c0000
[ 2.883169] CPU31: using LPI pending table @0x0000001fb57d0000
[ 2.883313] CPU31: Booted secondary processor 0x0000030303 [0x410fd082]
[ 2.965438] Detected PIPT I-cache on CPU32
[ 2.965479] GICv3: CPU32: found redistributor 50000 region
2:0x000004004d100000
[ 2.965541] CPU32: using LPI pending table @0x0000001fb57e0000
[ 2.965698] CPU32: Booted secondary processor 0x0000050000 [0x410fd082]
[ 3.048065] Detected PIPT I-cache on CPU33
[ 3.048089] GICv3: CPU33: found redistributor 50001 region
2:0x000004004d140000
[ 3.048145] CPU33: using LPI pending table @0x0000001fb5000000
[ 3.048252] CPU33: Booted secondary processor 0x0000050001 [0x410fd082]
[ 3.130711] Detected PIPT I-cache on CPU34
[ 3.130735] GICv3: CPU34: found redistributor 50002 region
2:0x000004004d180000
[ 3.130792] CPU34: using LPI pending table @0x0000001fb5010000
[ 3.130898] CPU34: Booted secondary processor 0x0000050002 [0x410fd082]
[ 3.213358] Detected PIPT I-cache on CPU35
[ 3.213383] GICv3: CPU35: found redistributor 50003 region
2:0x000004004d1c0000
[ 3.213439] CPU35: using LPI pending table @0x0000001fb5020000
[ 3.213545] CPU35: Booted secondary processor 0x0000050003 [0x410fd082]
[ 3.296005] Detected PIPT I-cache on CPU36
[ 3.296031] GICv3: CPU36: found redistributor 50100 region
2:0x000004004d200000
[ 3.296088] CPU36: using LPI pending table @0x0000001fb5030000
[ 3.296198] CPU36: Booted secondary processor 0x0000050100 [0x410fd082]
[ 3.378651] Detected PIPT I-cache on CPU37
[ 3.378676] GICv3: CPU37: found redistributor 50101 region
2:0x000004004d240000
[ 3.378733] CPU37: using LPI pending table @0x0000001fb5040000
[ 3.378840] CPU37: Booted secondary processor 0x0000050101 [0x410fd082]
[ 3.461298] Detected PIPT I-cache on CPU38
[ 3.461323] GICv3: CPU38: found redistributor 50102 region
2:0x000004004d280000
[ 3.461379] CPU38: using LPI pending table @0x0000001fb5050000
[ 3.461488] CPU38: Booted secondary processor 0x0000050102 [0x410fd082]
[ 3.543945] Detected PIPT I-cache on CPU39
[ 3.543970] GICv3: CPU39: found redistributor 50103 region
2:0x000004004d2c0000
[ 3.544028] CPU39: using LPI pending table @0x0000001fb5070000
[ 3.544134] CPU39: Booted secondary processor 0x0000050103 [0x410fd082]
[ 3.626593] Detected PIPT I-cache on CPU40
[ 3.626620] GICv3: CPU40: found redistributor 50200 region
2:0x000004004d300000
[ 3.626677] CPU40: using LPI pending table @0x0000001fb5080000
[ 3.626786] CPU40: Booted secondary processor 0x0000050200 [0x410fd082]
[ 3.709241] Detected PIPT I-cache on CPU41
[ 3.709266] GICv3: CPU41: found redistributor 50201 region
2:0x000004004d340000
[ 3.709323] CPU41: using LPI pending table @0x0000001fb5090000
[ 3.709428] CPU41: Booted secondary processor 0x0000050201 [0x410fd082]
[ 3.791888] Detected PIPT I-cache on CPU42
[ 3.791914] GICv3: CPU42: found redistributor 50202 region
2:0x000004004d380000
[ 3.791971] CPU42: using LPI pending table @0x0000001fb50b0000
[ 3.792078] CPU42: Booted secondary processor 0x0000050202 [0x410fd082]
[ 3.874536] Detected PIPT I-cache on CPU43
[ 3.874562] GICv3: CPU43: found redistributor 50203 region
2:0x000004004d3c0000
[ 3.874619] CPU43: using LPI pending table @0x0000001fb50c0000
[ 3.874726] CPU43: Booted secondary processor 0x0000050203 [0x410fd082]
[ 3.957185] Detected PIPT I-cache on CPU44
[ 3.957211] GICv3: CPU44: found redistributor 50300 region
2:0x000004004d400000
[ 3.957270] CPU44: using LPI pending table @0x0000001fb50d0000
[ 3.957378] CPU44: Booted secondary processor 0x0000050300 [0x410fd082]
[ 4.039832] Detected PIPT I-cache on CPU45
[ 4.039858] GICv3: CPU45: found redistributor 50301 region
2:0x000004004d440000
[ 4.039915] CPU45: using LPI pending table @0x0000001fb50f0000
[ 4.040022] CPU45: Booted secondary processor 0x0000050301 [0x410fd082]
[ 4.122477] Detected PIPT I-cache on CPU46
[ 4.122503] GICv3: CPU46: found redistributor 50302 region
2:0x000004004d480000
[ 4.122560] CPU46: using LPI pending table @0x0000001fb5100000
[ 4.122668] CPU46: Booted secondary processor 0x0000050302 [0x410fd082]
[ 4.205126] Detected PIPT I-cache on CPU47
[ 4.205152] GICv3: CPU47: found redistributor 50303 region
2:0x000004004d4c0000
[ 4.205209] CPU47: using LPI pending table @0x0000001fb5110000
[ 4.205316] CPU47: Booted secondary processor 0x0000050303 [0x410fd082]
[ 4.279425] Detected PIPT I-cache on CPU48
[ 4.279458] GICv3: CPU48: found redistributor 70000 region
3:0x000004006d100000
[ 4.279519] CPU48: using LPI pending table @0x0000001fb51b0000
[ 4.279669] CPU48: Booted secondary processor 0x0000070000 [0x410fd082]
[ 4.362072] Detected PIPT I-cache on CPU49
[ 4.362101] GICv3: CPU49: found redistributor 70001 region
3:0x000004006d140000
[ 4.362163] CPU49: using LPI pending table @0x0000001fb5250000
[ 4.362305] CPU49: Booted secondary processor 0x0000070001 [0x410fd082]
[ 4.444725] Detected PIPT I-cache on CPU50
[ 4.444753] GICv3: CPU50: found redistributor 70002 region
3:0x000004006d180000
[ 4.444817] CPU50: using LPI pending table @0x0000001fb5290000
[ 4.444962] CPU50: Booted secondary processor 0x0000070002 [0x410fd082]
[ 4.527375] Detected PIPT I-cache on CPU51
[ 4.527404] GICv3: CPU51: found redistributor 70003 region
3:0x000004006d1c0000
[ 4.527466] CPU51: using LPI pending table @0x0000001fb5310000
[ 4.527609] CPU51: Booted secondary processor 0x0000070003 [0x410fd082]
[ 4.610026] Detected PIPT I-cache on CPU52
[ 4.610058] GICv3: CPU52: found redistributor 70100 region
3:0x000004006d200000
[ 4.610122] CPU52: using LPI pending table @0x0000001fb5380000
[ 4.610273] CPU52: Booted secondary processor 0x0000070100 [0x410fd082]
[ 4.692677] Detected PIPT I-cache on CPU53
[ 4.692706] GICv3: CPU53: found redistributor 70101 region
3:0x000004006d240000
[ 4.692769] CPU53: using LPI pending table @0x0000001fb53d0000
[ 4.692912] CPU53: Booted secondary processor 0x0000070101 [0x410fd082]
[ 4.775327] Detected PIPT I-cache on CPU54
[ 4.775356] GICv3: CPU54: found redistributor 70102 region
3:0x000004006d280000
[ 4.775417] CPU54: using LPI pending table @0x0000001fb4c50000
[ 4.775561] CPU54: Booted secondary processor 0x0000070102 [0x410fd082]
[ 4.857978] Detected PIPT I-cache on CPU55
[ 4.858007] GICv3: CPU55: found redistributor 70103 region
3:0x000004006d2c0000
[ 4.858069] CPU55: using LPI pending table @0x0000001fb4cc0000
[ 4.858214] CPU55: Booted secondary processor 0x0000070103 [0x410fd082]
[ 4.940631] Detected PIPT I-cache on CPU56
[ 4.940663] GICv3: CPU56: found redistributor 70200 region
3:0x000004006d300000
[ 4.940726] CPU56: using LPI pending table @0x0000001fb4d10000
[ 4.940880] CPU56: Booted secondary processor 0x0000070200 [0x410fd082]
[ 5.023280] Detected PIPT I-cache on CPU57
[ 5.023309] GICv3: CPU57: found redistributor 70201 region
3:0x000004006d340000
[ 5.023372] CPU57: using LPI pending table @0x0000001fb4d80000
[ 5.023516] CPU57: Booted secondary processor 0x0000070201 [0x410fd082]
[ 5.105930] Detected PIPT I-cache on CPU58
[ 5.105959] GICv3: CPU58: found redistributor 70202 region
3:0x000004006d380000
[ 5.106021] CPU58: using LPI pending table @0x0000001fb4dc0000
[ 5.106166] CPU58: Booted secondary processor 0x0000070202 [0x410fd082]
[ 5.188582] Detected PIPT I-cache on CPU59
[ 5.188611] GICv3: CPU59: found redistributor 70203 region
3:0x000004006d3c0000
[ 5.188672] CPU59: using LPI pending table @0x0000001fb4e50000
[ 5.188815] CPU59: Booted secondary processor 0x0000070203 [0x410fd082]
[ 5.271235] Detected PIPT I-cache on CPU60
[ 5.271267] GICv3: CPU60: found redistributor 70300 region
3:0x000004006d400000
[ 5.271330] CPU60: using LPI pending table @0x0000001fb4ed0000
[ 5.271481] CPU60: Booted secondary processor 0x0000070300 [0x410fd082]
[ 5.353884] Detected PIPT I-cache on CPU61
[ 5.353913] GICv3: CPU61: found redistributor 70301 region
3:0x000004006d440000
[ 5.353979] CPU61: using LPI pending table @0x0000001fb4f00000
[ 5.354121] CPU61: Booted secondary processor 0x0000070301 [0x410fd082]
[ 5.436534] Detected PIPT I-cache on CPU62
[ 5.436563] GICv3: CPU62: found redistributor 70302 region
3:0x000004006d480000
[ 5.436626] CPU62: using LPI pending table @0x0000001fb4f90000
[ 5.436769] CPU62: Booted secondary processor 0x0000070302 [0x410fd082]
[ 5.519187] Detected PIPT I-cache on CPU63
[ 5.519217] GICv3: CPU63: found redistributor 70303 region
3:0x000004006d4c0000
[ 5.519280] CPU63: using LPI pending table @0x0000001fb4fc0000
[ 5.519421] CPU63: Booted secondary processor 0x0000070303 [0x410fd082]
[ 5.519506] smp: Brought up 4 nodes, 64 CPUs
[ 7.040874] SMP: Total of 64 processors activated.
[ 7.045720] CPU features: detected: GIC system register CPU interface
[ 7.052240] CPU features: detected: 32-bit EL0 Support
[ 7.144026] CPU: All CPU(s) started at EL2
[ 7.148298] alternatives: patching kernel code
[ 7.155277] Unable to handle kernel NULL pointer dereference at
virtual address 0000000000000000
[ 7.164163] Mem abort info:
[ 7.166978] ESR = 0x96000004
[ 7.170061] Exception class = DABT (current EL), IL = 32 bits
[ 7.176043] SET = 0, FnV = 0
[ 7.179121] EA = 0, S1PTW = 0
[ 7.182291] Data abort info:
[ 7.185193] ISV = 0, ISS = 0x00000004
[ 7.189066] CM = 0, WnR = 0
[ 7.192056] [0000000000000000] user address but active_mm is swapper
[ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 7.204107] Modules linked in:
[ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
4.19.0-00002-g3ed52fd-dirty #807
[ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
[ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
[ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
[ 7.237797] sp : ffff00000947bc60
[ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
[ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
[ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
[ 7.257235] x23: 0000000000000010 x22: 0000000000000001
[ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
[ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
[ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
[ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
[ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
[ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
[ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
[ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
[ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
[ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
[ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
[ 7.321613] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
[ 7.328382] Call trace:
[ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
[ 7.335420] destroy_sched_domain+0x20/0x70
[ 7.339642] cpu_attach_domain+0xc8/0x2e8
[ 7.343687] build_sched_domains+0xd44/0xde0
[ 7.347996] sched_init_domains+0x68/0x88
[ 7.352044] sched_init_smp+0x2c/0x7c
[ 7.355738] kernel_init_freeable+0xdc/0x244
[ 7.360049] kernel_init+0x10/0x108
[ 7.363567] ret_from_fork+0x10/0x18
[ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
[ 7.373400] ---[ end trace 8150af869a14363f ]---
[ 7.378076] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 7.378076]
[ 7.387320] SMP: stopping secondary CPUs
[ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b
[ 7.391297] ]---
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-10-23 10:30 Crash report: Broken NUMA distance map causes crash on arm64 system John Garry
@ 2018-10-25 11:01 ` John Garry
2018-10-30 9:26 ` Peter Zijlstra
1 sibling, 0 replies; 13+ messages in thread
From: John Garry @ 2018-10-25 11:01 UTC (permalink / raw)
To: linux-arm-kernel, linux-kernel, Will Deacon, Catalin Marinas,
Rob Herring, devicetree, Frank Rowand, Ingo Molnar,
Peter Zijlstra, suravee.suthikulpanit
Cc: Linuxarm
On 23/10/2018 11:30, John Garry wrote:
> Hi all,
>
> I have stumbled upon this crash on my arm64 system:
>
JFYI, I bisected this issue to the following patch:
commit 58d5af59d55b4e9f3015207afa6a6648e9b005b1
Author: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Date: Thu Sep 7 02:20:05 2017 -0500
sched/topology: Introduce NUMA identity node sched domain
On AMD Family17h-based (EPYC) system, a logical NUMA node can contain
upto 8 cores (16 threads) with the following topology.
which either exposes or causes this issue.
John
> [ 7.040874] SMP: Total of 64 processors activated.
> [ 7.045720] CPU features: detected: GIC system register CPU interface
> [ 7.052240] CPU features: detected: 32-bit EL0 Support
> [ 7.144026] CPU: All CPU(s) started at EL2
> [ 7.148298] alternatives: patching kernel code
> [ 7.155277] Unable to handle kernel NULL pointer dereference at
> virtual address 0000000000000000
> [ 7.164163] Mem abort info:
> [ 7.166978] ESR = 0x96000004
> [ 7.170061] Exception class = DABT (current EL), IL = 32 bits
> [ 7.176043] SET = 0, FnV = 0
> [ 7.179121] EA = 0, S1PTW = 0
> [ 7.182291] Data abort info:
> [ 7.185193] ISV = 0, ISS = 0x00000004
> [ 7.189066] CM = 0, WnR = 0
> [ 7.192056] [0000000000000000] user address but active_mm is swapper
> [ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
> [ 7.204107] Modules linked in:
> [ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
> 4.19.0-00002-g3ed52fd-dirty #807
> [ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
> [ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
> [ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
> [ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
> [ 7.237797] sp : ffff00000947bc60
> [ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
> [ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
> [ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
> [ 7.257235] x23: 0000000000000010 x22: 0000000000000001
> [ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
> [ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
> [ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
> [ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
> [ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
> [ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
> [ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
> [ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
> [ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
> [ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
> [ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
> [ 7.321613] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
> [ 7.328382] Call trace:
> [ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
> [ 7.335420] destroy_sched_domain+0x20/0x70
> [ 7.339642] cpu_attach_domain+0xc8/0x2e8
> [ 7.343687] build_sched_domains+0xd44/0xde0
> [ 7.347996] sched_init_domains+0x68/0x88
> [ 7.352044] sched_init_smp+0x2c/0x7c
> [ 7.355738] kernel_init_freeable+0xdc/0x244
> [ 7.360049] kernel_init+0x10/0x108
> [ 7.363567] ret_from_fork+0x10/0x18
> [ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
> [ 7.373400] ---[ end trace 8150af869a14363f ]---
> [ 7.378076] Kernel panic - not syncing: Attempted to kill init!
> exitcode=0x0000000b
> [ 7.378076]
> [ 7.387320] SMP: stopping secondary CPUs
> [ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill
> init! exitcode=0x0000000b
> [ 7.391297] ]---
>
> I will straightaway note that I have been fiddling with my board's
> device tree, specifically (breaking!) the NUMA distance map, like this:
>
> distance-matrix = <0 0 10>,
> <0 1 15>,
> <0 2 20>,
> <0 3 25>,
> <1 0 15>,
> <1 1 10>,
> <1 2 25>,
> <1 3 30>,
> <2 0 20>,
> <2 1 25>,
> <2 2 10>,
> <2 3 15>,
> <3 0 10>,* should be same as 0->3 and > 10
> <3 1 10>,* should be same as 1->3 and > 10
> <3 2 15>,
> <3 3 15>;* should be 10
>
> However I don't think that this should crash the kernel. I'd say
> of_numa_parse_distance_map_v1() should robustly handle broken maps,
> probably by erroring and causing NUMA to be disabled.
>
> However I am interested to know if the scheduler crash is a real problem.
>
> The kernel is 4.19. Full log is below.
>
> Thanks,
> John
>
> Full log:
>
> EFI stub: Booting Linux Kernel...
> SmiGraphicsOutputQueryMode +
> SmiGraphicsOutputQueryMode -
> SmiGraphicsOutputQueryMode +
> SmiGraphicsOutputQueryMode -
> EFI stub: Using DTB from configuration table
> EFI stub: Exiting boot services and installing virtual address map...
> [SAL_ClearAffiliationSMP,325]it is going to hard reset
> dev:0x500E004AAAAAAA1F
> SAS ExitBootServicesEvent
> GMAC ExitBootServicesEvent
> GMAC ExitBootServicesEvent
> GMAC ExitBootServicesEvent
> GMAC ExitBootServicesEvent
> OHCI ExitBootServicesEvent
> IPMI ExitBootService Event
> TransferSmbiosToBMC EVENT.
> Transfer Smbios Table To iBMC Success.
> GetVariable Status : Not Found.
> [ 0.000000] Booting Linux on physical CPU 0x0000010000 [0x410fd082]
> [ 0.000000] Linux version 4.19.0-00002-g3ed52fd-dirty
> (johnpgarry@johnpgarry-ThinkCentre-M93p) (gcc version 7.3.1 20180425
> [linaro-7.3-2018.05-rc1 revision
> 38aec9a676236eaa42ca03ccb3a6c1dd0182c29f] (Linaro GCC 7.3-2018.05-rc1))
> #807 SMP PREEMPT Tue Oct 23 11:02:19 BST 2018
> [ 0.000000] Machine model: Hisilicon Hip07 D05 Development Board
> [ 0.000000] earlycon: pl11 at MMIO 0x00000000602b0000 (options '')
> [ 0.000000] bootconsole [pl11] enabled
> [ 0.000000] efi: Getting EFI parameters from FDT:
> [ 0.000000] efi: EFI v2.60 by EDK II
> [ 0.000000] efi: SMBIOS=0x3f050000 SMBIOS 3.0=0x39b00000
> ACPI=0x39bd0000 ACPI 2.0=0x39bd0014 MEMATTR=0x3cc99018
> [ 0.000000] crashkernel reserved: 0x0000000002000000 -
> 0x0000000012000000 (256 MB)
> [ 0.000000] cma: Reserved 32 MiB at 0x000000003d000000
> [ 0.000000] earlycon: pl11 at MMIO32 0x00000000602b0000 (options
> '115200n8')
> [ 0.000000] ------------[ cut here ]------------
> [ 0.000000] console 'pl11' already registered
> [ 0.000000] WARNING: CPU: 0 PID: 0 at kernel/printk/printk.c:2586
> register_console+0x58/0x3c8
> [ 0.000000] Modules linked in:
> [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
> 4.19.0-00002-g3ed52fd-dirty #807
> [ 0.000000] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
> [ 0.000000] pstate: 60000085 (nZCv daIf -PAN -UAO)
> [ 0.000000] pc : register_console+0x58/0x3c8
> [ 0.000000] lr : register_console+0x58/0x3c8
> [ 0.000000] sp : ffff000009103de0
> [ 0.000000] x29: ffff000009103de0 x28: 0000000000fb0018
> [ 0.000000] x27: 000000003cc81018 x26: ffff0000090d1098
> [ 0.000000] x25: 0000000000004b10 x24: ffff0000090e4348
> [ 0.000000] x23: ffff0000090d1098 x22: ffff000009262000
> [ 0.000000] x21: ffff0000091fe150 x20: 0000000000004b10
> [ 0.000000] x19: ffff0000091fe150 x18: ffffffffffffffff
> [ 0.000000] x17: 0000000000000000 x16: 0000000000000000
> [ 0.000000] x15: ffff0000091096c8 x14: ffff0000892626bf
> [ 0.000000] x13: ffff0000092626cd x12: ffff000009109940
> [ 0.000000] x11: ffff0000085ecd98 x10: ffff000009103ab0
> [ 0.000000] x9 : 000000000000000d x8 : 7369676572207964
> [ 0.000000] x7 : 6165726c61202731 x6 : 000000000000000d
> [ 0.000000] x5 : 0000000000000000 x4 : 0000000000000000
> [ 0.000000] x3 : ffffffffffffffff x2 : ffff0000091210f8
> [ 0.000000] x1 : 0000000000000000 x0 : 0000000000000000
> [ 0.000000] Call trace:
> [ 0.000000] register_console+0x58/0x3c8
> [ 0.000000] of_setup_earlycon+0x20c/0x22c
> [ 0.000000] early_init_dt_scan_chosen_stdout+0x158/0x19c
> [ 0.000000] acpi_boot_table_init+0x150/0x184
> [ 0.000000] setup_arch+0x1cc/0x530
> [ 0.000000] start_kernel+0x80/0x418
> [ 0.000000] random: get_random_bytes called from
> print_oops_end_marker+0x48/0x68 with crng_init=0
> [ 0.000000] ---[ end trace 0000000000000000 ]---
> [ 0.000000] NUMA: NODE_DATA [mem 0x1ffbffe840-0x1ffbffffff]
> [ 0.000000] NUMA: Initmem setup node 1 [<memory-less node>]
> [ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf41fc0-0x41ffbf4377f]
> [ 0.000000] NUMA: NODE_DATA(1) on node 2
> [ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf40800-0x41ffbf41fbf]
> [ 0.000000] NUMA: Initmem setup node 3 [<memory-less node>]
> [ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf3f040-0x41ffbf407ff]
> [ 0.000000] NUMA: NODE_DATA(3) on node 2
> [ 0.000000] Zone ranges:
> [ 0.000000] DMA32 [mem 0x0000000000000000-0x00000000ffffffff]
> [ 0.000000] Normal [mem 0x0000000100000000-0x0000041ffbffffff]
> [ 0.000000] Movable zone start for each node
> [ 0.000000] Early memory node ranges
> [ 0.000000] node 0: [mem 0x0000000000000000-0x0000000031902fff]
> [ 0.000000] node 0: [mem 0x0000000031903000-0x0000000031906fff]
> [ 0.000000] node 0: [mem 0x0000000031907000-0x000000003999ffff]
> [ 0.000000] node 0: [mem 0x00000000399a0000-0x0000000039a3ffff]
> [ 0.000000] node 0: [mem 0x0000000039a40000-0x0000000039a8ffff]
> [ 0.000000] node 0: [mem 0x0000000039a90000-0x0000000039a9ffff]
> [ 0.000000] node 0: [mem 0x0000000039aa0000-0x0000000039aeffff]
> [ 0.000000] node 0: [mem 0x0000000039af0000-0x0000000039bbffff]
> [ 0.000000] node 0: [mem 0x0000000039bc0000-0x0000000039bdffff]
> [ 0.000000] node 0: [mem 0x0000000039be0000-0x000000003a05ffff]
> [ 0.000000] node 0: [mem 0x000000003a060000-0x000000003f04ffff]
> [ 0.000000] node 0: [mem 0x000000003f050000-0x000000003f07ffff]
> [ 0.000000] node 0: [mem 0x000000003f080000-0x000000003fbfffff]
> [ 0.000000] node 0: [mem 0x0000001040000000-0x0000001ffbffffff]
> [ 0.000000] node 2: [mem 0x0000041000000000-0x0000041ffbffffff]
> [ 0.000000] Initmem setup node 0 [mem
> 0x0000000000000000-0x0000001ffbffffff]
> [ 0.000000] Could not find start_pfn for node 1
> [ 0.000000] Initmem setup node 1 [mem
> 0x0000000000000000-0x0000000000000000]
> [ 0.000000] Initmem setup node 2 [mem
> 0x0000041000000000-0x0000041ffbffffff]
> [ 0.000000] Could not find start_pfn for node 3
> [ 0.000000] Initmem setup node 3 [mem
> 0x0000000000000000-0x0000000000000000]
> [ 0.000000] psci: probing for conduit method from DT.
> [ 0.000000] percpu: Embedded 23 pages/cpu @(____ptrval____) s56408
> r8192 d29608 u94208
> [ 0.000000] Detected PIPT I-cache on CPU0
> [ 0.000000] CPU features: enabling workaround for EL2 vector hardening
> [ 0.000000] CPU features: detected: Kernel page table isolation (KPTI)
> [ 0.000000] Built 4 zonelists, mobility grouping on. Total pages:
> 32996880
> [ 0.000000] Policy zone: Normal
> [ 0.000000] Kernel command line: BOOT_IMAGE=/john/Image rdinit=/init
> crashkernel=256M@32M console=ttyAMA0,115200
> earlycon=pl011,mmio,0x602b0000 acpi=off pcie_aspm=off earlycon
> [ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
> [ 0.000000] log_buf_len total cpu_extra contributions: 258048 bytes
> [ 0.000000] log_buf_len min size: 131072 bytes
> [ 0.000000] log_buf_len: 524288 bytes
> [ 0.000000] early log buf free: 123488(94%)
> [ 0.000000] software IO TLB: mapped [mem 0x359a0000-0x399a0000] (64MB)
> [ 0.000000] Memory: 131332260K/134082560K available (10620K kernel
> code, 1370K rwdata, 4888K rodata, 1344K init, 380K bss, 2717532K
> reserved, 32768K cma-reserved)
> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=64, Nodes=4
> [ 0.000000] rcu: Preemptible hierarchical RCU implementation.
> [ 0.000000] Tasks RCU enabled.
> [ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
> [ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
> [ 0.000000] GICv3: Distributor has no Range Selector support
> [ 0.000000] GICv3: VLPI support, direct LPI support
> [ 0.000000] ITS [mem 0x4c000000-0x4c03ffff]
> [ 0.000000] ITS@0x000000004c000000: Using ITS number 0
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x000000004c000000: allocated 524288 Devices
> @1fbb000000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x000000004c000000: allocated 65536 Virtual CPUs
> @1fbb980000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x000000004c000000: allocated 512 Interrupt
> Collections @1fbb929000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0x6c000000-0x6c03ffff]
> [ 0.000000] ITS@0x000000006c000000: Using ITS number 1
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x000000006c000000: allocated 524288 Devices
> @1fbac00000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x000000006c000000: allocated 65536 Virtual CPUs
> @1fbba00000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x000000006c000000: allocated 512 Interrupt
> Collections @1fbb92a000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0xc6000000-0xc603ffff]
> [ 0.000000] ITS@0x00000000c6000000: Using ITS number 2
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x00000000c6000000: allocated 524288 Devices
> @1fba800000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x00000000c6000000: allocated 65536 Virtual CPUs
> @1fbba80000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt
> Collections @1fbb92c000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0x8c6000000-0x8c603ffff]
> [ 0.000000] ITS@0x00000008c6000000: Using ITS number 3
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x00000008c6000000: allocated 524288 Devices
> @1fba400000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x00000008c6000000: allocated 65536 Virtual CPUs
> @1fbbb00000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x00000008c6000000: allocated 512 Interrupt
> Collections @1fbb92e000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0x4004c000000-0x4004c03ffff]
> [ 0.000000] ITS@0x000004004c000000: Using ITS number 4
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x000004004c000000: allocated 524288 Devices
> @1fba000000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x000004004c000000: allocated 65536 Virtual CPUs
> @1fbbb80000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x000004004c000000: allocated 512 Interrupt
> Collections @1fbb92f000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0x4006c000000-0x4006c03ffff]
> [ 0.000000] ITS@0x000004006c000000: Using ITS number 5
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x000004006c000000: allocated 524288 Devices
> @1fb9800000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x000004006c000000: allocated 65536 Virtual CPUs
> @1fb9c80000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x000004006c000000: allocated 512 Interrupt
> Collections @1fb9c10000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0x400c6000000-0x400c603ffff]
> [ 0.000000] ITS@0x00000400c6000000: Using ITS number 6
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x00000400c6000000: allocated 524288 Devices
> @1fb9400000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x00000400c6000000: allocated 65536 Virtual CPUs
> @1fb9d00000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x00000400c6000000: allocated 512 Interrupt
> Collections @1fb9c11000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS [mem 0x408c6000000-0x408c603ffff]
> [ 0.000000] ITS@0x00000408c6000000: Using ITS number 7
> [ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
> [ 0.000000] ITS@0x00000408c6000000: allocated 524288 Devices
> @1fb9000000 (flat, esz 8, psz 16K, shr 1)
> [ 0.000000] ITS@0x00000408c6000000: allocated 65536 Virtual CPUs
> @1fb9d80000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] ITS@0x00000408c6000000: allocated 512 Interrupt
> Collections @1fb9c13000 (flat, esz 8, psz 4K, shr 1)
> [ 0.000000] GIC: using LPI property table @0x0000001fb9c40000
> [ 0.000000] ITS: Using DirectLPI for VPE invalidation
> [ 0.000000] ITS: Enabling GICv4 support
> [ 0.000000] GICv3: CPU0: found redistributor 10000 region
> 0:0x000000004d100000
> [ 0.000000] CPU0: using LPI pending table @0x0000001fb9c50000
> [ 0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
> [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff
> max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
> [ 0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps
> every 4398046511100ns
> [ 0.008248] Console: colour dummy device 80x25
> [ 0.012791] mempolicy: Enabling automatic NUMA balancing. Configure
> with numa_balancing= or the kernel.numa_balancing sysctl
> [ 0.024159] Calibrating delay loop (skipped), value calculated using
> timer frequency.. 100.00 BogoMIPS (lpj=200000)
> [ 0.034713] pid_max: default: 65536 minimum: 512
> [ 0.039438] Security Framework initialized
> [ 0.065597] Dentry cache hash table entries: 8388608 (order: 14,
> 67108864 bytes)
> [ 0.084013] Inode-cache hash table entries: 4194304 (order: 13,
> 33554432 bytes)
> [ 0.091758] Mount-cache hash table entries: 131072 (order: 8, 1048576
> bytes)
> [ 0.099226] Mountpoint-cache hash table entries: 131072 (order: 8,
> 1048576 bytes)
> [ 0.150842] ASID allocator initialised with 32768 entries
> [ 0.164305] rcu: Hierarchical SRCU implementation.
> [ 0.177184] Platform MSI: interrupt-controller@4c000000 domain created
> [ 0.183784] Platform MSI: interrupt-controller@6c000000 domain created
> [ 0.190384] Platform MSI: interrupt-controller@c6000000 domain created
> [ 0.196983] Platform MSI: interrupt-controller@8,c6000000 domain created
> [ 0.203758] Platform MSI: interrupt-controller@400,4c000000 domain
> created
> [ 0.210708] Platform MSI: interrupt-controller@400,6c000000 domain
> created
> [ 0.217659] Platform MSI: interrupt-controller@400,c6000000 domain
> created
> [ 0.224610] Platform MSI: interrupt-controller@408,c6000000 domain
> created
> [ 0.231606] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@4c000000 domain created
> [ 0.240491] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@6c000000 domain created
> [ 0.249375] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@c6000000 domain created
> [ 0.258260] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@8,c6000000 domain
> created
> [ 0.267320] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@400,4c000000 domain
> created
> [ 0.276556] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@400,6c000000 domain
> created
> [ 0.285792] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@400,c6000000 domain
> created
> [ 0.295029] PCI/MSI:
> /interrupt-controller@4d000000/interrupt-controller@408,c6000000 domain
> created
> [ 0.304714] Remapping and enabling EFI services.
> [ 0.317387] smp: Bringing up secondary CPUs ...
> [ 0.404570] Detected PIPT I-cache on CPU1
> [ 0.404582] GICv3: CPU1: found redistributor 10001 region
> 0:0x000000004d140000
> [ 0.404611] CPU1: using LPI pending table @0x0000001fb5dc0000
> [ 0.404728] CPU1: Booted secondary processor 0x0000010001 [0x410fd082]
> [ 0.487192] Detected PIPT I-cache on CPU2
> [ 0.487198] GICv3: CPU2: found redistributor 10002 region
> 0:0x000000004d180000
> [ 0.487224] CPU2: using LPI pending table @0x0000001fb5e10000
> [ 0.487329] CPU2: Booted secondary processor 0x0000010002 [0x410fd082]
> [ 0.569818] Detected PIPT I-cache on CPU3
> [ 0.569824] GICv3: CPU3: found redistributor 10003 region
> 0:0x000000004d1c0000
> [ 0.569851] CPU3: using LPI pending table @0x0000001fb5e30000
> [ 0.569953] CPU3: Booted secondary processor 0x0000010003 [0x410fd082]
> [ 0.652445] Detected PIPT I-cache on CPU4
> [ 0.652453] GICv3: CPU4: found redistributor 10100 region
> 0:0x000000004d200000
> [ 0.652480] CPU4: using LPI pending table @0x0000001fb5e40000
> [ 0.652584] CPU4: Booted secondary processor 0x0000010100 [0x410fd082]
> [ 0.735070] Detected PIPT I-cache on CPU5
> [ 0.735077] GICv3: CPU5: found redistributor 10101 region
> 0:0x000000004d240000
> [ 0.735103] CPU5: using LPI pending table @0x0000001fb5e90000
> [ 0.735206] CPU5: Booted secondary processor 0x0000010101 [0x410fd082]
> [ 0.817697] Detected PIPT I-cache on CPU6
> [ 0.817704] GICv3: CPU6: found redistributor 10102 region
> 0:0x000000004d280000
> [ 0.817731] CPU6: using LPI pending table @0x0000001fb5ea0000
> [ 0.817834] CPU6: Booted secondary processor 0x0000010102 [0x410fd082]
> [ 0.900324] Detected PIPT I-cache on CPU7
> [ 0.900331] GICv3: CPU7: found redistributor 10103 region
> 0:0x000000004d2c0000
> [ 0.900358] CPU7: using LPI pending table @0x0000001fb5eb0000
> [ 0.900461] CPU7: Booted secondary processor 0x0000010103 [0x410fd082]
> [ 0.982952] Detected PIPT I-cache on CPU8
> [ 0.982961] GICv3: CPU8: found redistributor 10200 region
> 0:0x000000004d300000
> [ 0.982988] CPU8: using LPI pending table @0x0000001fb5f00000
> [ 0.983092] CPU8: Booted secondary processor 0x0000010200 [0x410fd082]
> [ 1.065578] Detected PIPT I-cache on CPU9
> [ 1.065585] GICv3: CPU9: found redistributor 10201 region
> 0:0x000000004d340000
> [ 1.065611] CPU9: using LPI pending table @0x0000001fb5f20000
> [ 1.065714] CPU9: Booted secondary processor 0x0000010201 [0x410fd082]
> [ 1.148205] Detected PIPT I-cache on CPU10
> [ 1.148213] GICv3: CPU10: found redistributor 10202 region
> 0:0x000000004d380000
> [ 1.148240] CPU10: using LPI pending table @0x0000001fb5f30000
> [ 1.148343] CPU10: Booted secondary processor 0x0000010202 [0x410fd082]
> [ 1.230833] Detected PIPT I-cache on CPU11
> [ 1.230841] GICv3: CPU11: found redistributor 10203 region
> 0:0x000000004d3c0000
> [ 1.230868] CPU11: using LPI pending table @0x0000001fb5f80000
> [ 1.230971] CPU11: Booted secondary processor 0x0000010203 [0x410fd082]
> [ 1.313460] Detected PIPT I-cache on CPU12
> [ 1.313469] GICv3: CPU12: found redistributor 10300 region
> 0:0x000000004d400000
> [ 1.313496] CPU12: using LPI pending table @0x0000001fb5fa0000
> [ 1.313602] CPU12: Booted secondary processor 0x0000010300 [0x410fd082]
> [ 1.396086] Detected PIPT I-cache on CPU13
> [ 1.396094] GICv3: CPU13: found redistributor 10301 region
> 0:0x000000004d440000
> [ 1.396121] CPU13: using LPI pending table @0x0000001fb5fb0000
> [ 1.396226] CPU13: Booted secondary processor 0x0000010301 [0x410fd082]
> [ 1.478713] Detected PIPT I-cache on CPU14
> [ 1.478722] GICv3: CPU14: found redistributor 10302 region
> 0:0x000000004d480000
> [ 1.478748] CPU14: using LPI pending table @0x0000001fb5fc0000
> [ 1.478852] CPU14: Booted secondary processor 0x0000010302 [0x410fd082]
> [ 1.561340] Detected PIPT I-cache on CPU15
> [ 1.561348] GICv3: CPU15: found redistributor 10303 region
> 0:0x000000004d4c0000
> [ 1.561374] CPU15: using LPI pending table @0x0000001fb5820000
> [ 1.561477] CPU15: Booted secondary processor 0x0000010303 [0x410fd082]
> [ 1.643629] Detected PIPT I-cache on CPU16
> [ 1.643641] GICv3: CPU16: found redistributor 30000 region
> 1:0x000000006d100000
> [ 1.643680] CPU16: using LPI pending table @0x0000001fb5860000
> [ 1.643828] CPU16: Booted secondary processor 0x0000030000 [0x410fd082]
> [ 1.726259] Detected PIPT I-cache on CPU17
> [ 1.726268] GICv3: CPU17: found redistributor 30001 region
> 1:0x000000006d140000
> [ 1.726305] CPU17: using LPI pending table @0x0000001fb58f0000
> [ 1.726449] CPU17: Booted secondary processor 0x0000030001 [0x410fd082]
> [ 1.808890] Detected PIPT I-cache on CPU18
> [ 1.808900] GICv3: CPU18: found redistributor 30002 region
> 1:0x000000006d180000
> [ 1.808936] CPU18: using LPI pending table @0x0000001fb5970000
> [ 1.809080] CPU18: Booted secondary processor 0x0000030002 [0x410fd082]
> [ 1.891523] Detected PIPT I-cache on CPU19
> [ 1.891533] GICv3: CPU19: found redistributor 30003 region
> 1:0x000000006d1c0000
> [ 1.891570] CPU19: using LPI pending table @0x0000001fb59b0000
> [ 1.891713] CPU19: Booted secondary processor 0x0000030003 [0x410fd082]
> [ 1.974158] Detected PIPT I-cache on CPU20
> [ 1.974171] GICv3: CPU20: found redistributor 30100 region
> 1:0x000000006d200000
> [ 1.974209] CPU20: using LPI pending table @0x0000001fb5a20000
> [ 1.974356] CPU20: Booted secondary processor 0x0000030100 [0x410fd082]
> [ 2.056788] Detected PIPT I-cache on CPU21
> [ 2.056797] GICv3: CPU21: found redistributor 30101 region
> 1:0x000000006d240000
> [ 2.056834] CPU21: using LPI pending table @0x0000001fb5ab0000
> [ 2.056979] CPU21: Booted secondary processor 0x0000030101 [0x410fd082]
> [ 2.139420] Detected PIPT I-cache on CPU22
> [ 2.139430] GICv3: CPU22: found redistributor 30102 region
> 1:0x000000006d280000
> [ 2.139466] CPU22: using LPI pending table @0x0000001fb5af0000
> [ 2.139610] CPU22: Booted secondary processor 0x0000030102 [0x410fd082]
> [ 2.222053] Detected PIPT I-cache on CPU23
> [ 2.222063] GICv3: CPU23: found redistributor 30103 region
> 1:0x000000006d2c0000
> [ 2.222100] CPU23: using LPI pending table @0x0000001fb5b70000
> [ 2.222244] CPU23: Booted secondary processor 0x0000030103 [0x410fd082]
> [ 2.304688] Detected PIPT I-cache on CPU24
> [ 2.304701] GICv3: CPU24: found redistributor 30200 region
> 1:0x000000006d300000
> [ 2.304739] CPU24: using LPI pending table @0x0000001fb5bb0000
> [ 2.304888] CPU24: Booted secondary processor 0x0000030200 [0x410fd082]
> [ 2.387319] Detected PIPT I-cache on CPU25
> [ 2.387329] GICv3: CPU25: found redistributor 30201 region
> 1:0x000000006d340000
> [ 2.387365] CPU25: using LPI pending table @0x0000001fb5430000
> [ 2.387510] CPU25: Booted secondary processor 0x0000030201 [0x410fd082]
> [ 2.469951] Detected PIPT I-cache on CPU26
> [ 2.469961] GICv3: CPU26: found redistributor 30202 region
> 1:0x000000006d380000
> [ 2.469998] CPU26: using LPI pending table @0x0000001fb54a0000
> [ 2.470143] CPU26: Booted secondary processor 0x0000030202 [0x410fd082]
> [ 2.552585] Detected PIPT I-cache on CPU27
> [ 2.552595] GICv3: CPU27: found redistributor 30203 region
> 1:0x000000006d3c0000
> [ 2.552632] CPU27: using LPI pending table @0x0000001fb54f0000
> [ 2.552777] CPU27: Booted secondary processor 0x0000030203 [0x410fd082]
> [ 2.635220] Detected PIPT I-cache on CPU28
> [ 2.635234] GICv3: CPU28: found redistributor 30300 region
> 1:0x000000006d400000
> [ 2.635271] CPU28: using LPI pending table @0x0000001fb5570000
> [ 2.635419] CPU28: Booted secondary processor 0x0000030300 [0x410fd082]
> [ 2.717851] Detected PIPT I-cache on CPU29
> [ 2.717861] GICv3: CPU29: found redistributor 30301 region
> 1:0x000000006d440000
> [ 2.717898] CPU29: using LPI pending table @0x0000001fb55b0000
> [ 2.718043] CPU29: Booted secondary processor 0x0000030301 [0x410fd082]
> [ 2.800486] Detected PIPT I-cache on CPU30
> [ 2.800496] GICv3: CPU30: found redistributor 30302 region
> 1:0x000000006d480000
> [ 2.800533] CPU30: using LPI pending table @0x0000001fb5790000
> [ 2.800679] CPU30: Booted secondary processor 0x0000030302 [0x410fd082]
> [ 2.883122] Detected PIPT I-cache on CPU31
> [ 2.883133] GICv3: CPU31: found redistributor 30303 region
> 1:0x000000006d4c0000
> [ 2.883169] CPU31: using LPI pending table @0x0000001fb57d0000
> [ 2.883313] CPU31: Booted secondary processor 0x0000030303 [0x410fd082]
> [ 2.965438] Detected PIPT I-cache on CPU32
> [ 2.965479] GICv3: CPU32: found redistributor 50000 region
> 2:0x000004004d100000
> [ 2.965541] CPU32: using LPI pending table @0x0000001fb57e0000
> [ 2.965698] CPU32: Booted secondary processor 0x0000050000 [0x410fd082]
> [ 3.048065] Detected PIPT I-cache on CPU33
> [ 3.048089] GICv3: CPU33: found redistributor 50001 region
> 2:0x000004004d140000
> [ 3.048145] CPU33: using LPI pending table @0x0000001fb5000000
> [ 3.048252] CPU33: Booted secondary processor 0x0000050001 [0x410fd082]
> [ 3.130711] Detected PIPT I-cache on CPU34
> [ 3.130735] GICv3: CPU34: found redistributor 50002 region
> 2:0x000004004d180000
> [ 3.130792] CPU34: using LPI pending table @0x0000001fb5010000
> [ 3.130898] CPU34: Booted secondary processor 0x0000050002 [0x410fd082]
> [ 3.213358] Detected PIPT I-cache on CPU35
> [ 3.213383] GICv3: CPU35: found redistributor 50003 region
> 2:0x000004004d1c0000
> [ 3.213439] CPU35: using LPI pending table @0x0000001fb5020000
> [ 3.213545] CPU35: Booted secondary processor 0x0000050003 [0x410fd082]
> [ 3.296005] Detected PIPT I-cache on CPU36
> [ 3.296031] GICv3: CPU36: found redistributor 50100 region
> 2:0x000004004d200000
> [ 3.296088] CPU36: using LPI pending table @0x0000001fb5030000
> [ 3.296198] CPU36: Booted secondary processor 0x0000050100 [0x410fd082]
> [ 3.378651] Detected PIPT I-cache on CPU37
> [ 3.378676] GICv3: CPU37: found redistributor 50101 region
> 2:0x000004004d240000
> [ 3.378733] CPU37: using LPI pending table @0x0000001fb5040000
> [ 3.378840] CPU37: Booted secondary processor 0x0000050101 [0x410fd082]
> [ 3.461298] Detected PIPT I-cache on CPU38
> [ 3.461323] GICv3: CPU38: found redistributor 50102 region
> 2:0x000004004d280000
> [ 3.461379] CPU38: using LPI pending table @0x0000001fb5050000
> [ 3.461488] CPU38: Booted secondary processor 0x0000050102 [0x410fd082]
> [ 3.543945] Detected PIPT I-cache on CPU39
> [ 3.543970] GICv3: CPU39: found redistributor 50103 region
> 2:0x000004004d2c0000
> [ 3.544028] CPU39: using LPI pending table @0x0000001fb5070000
> [ 3.544134] CPU39: Booted secondary processor 0x0000050103 [0x410fd082]
> [ 3.626593] Detected PIPT I-cache on CPU40
> [ 3.626620] GICv3: CPU40: found redistributor 50200 region
> 2:0x000004004d300000
> [ 3.626677] CPU40: using LPI pending table @0x0000001fb5080000
> [ 3.626786] CPU40: Booted secondary processor 0x0000050200 [0x410fd082]
> [ 3.709241] Detected PIPT I-cache on CPU41
> [ 3.709266] GICv3: CPU41: found redistributor 50201 region
> 2:0x000004004d340000
> [ 3.709323] CPU41: using LPI pending table @0x0000001fb5090000
> [ 3.709428] CPU41: Booted secondary processor 0x0000050201 [0x410fd082]
> [ 3.791888] Detected PIPT I-cache on CPU42
> [ 3.791914] GICv3: CPU42: found redistributor 50202 region
> 2:0x000004004d380000
> [ 3.791971] CPU42: using LPI pending table @0x0000001fb50b0000
> [ 3.792078] CPU42: Booted secondary processor 0x0000050202 [0x410fd082]
> [ 3.874536] Detected PIPT I-cache on CPU43
> [ 3.874562] GICv3: CPU43: found redistributor 50203 region
> 2:0x000004004d3c0000
> [ 3.874619] CPU43: using LPI pending table @0x0000001fb50c0000
> [ 3.874726] CPU43: Booted secondary processor 0x0000050203 [0x410fd082]
> [ 3.957185] Detected PIPT I-cache on CPU44
> [ 3.957211] GICv3: CPU44: found redistributor 50300 region
> 2:0x000004004d400000
> [ 3.957270] CPU44: using LPI pending table @0x0000001fb50d0000
> [ 3.957378] CPU44: Booted secondary processor 0x0000050300 [0x410fd082]
> [ 4.039832] Detected PIPT I-cache on CPU45
> [ 4.039858] GICv3: CPU45: found redistributor 50301 region
> 2:0x000004004d440000
> [ 4.039915] CPU45: using LPI pending table @0x0000001fb50f0000
> [ 4.040022] CPU45: Booted secondary processor 0x0000050301 [0x410fd082]
> [ 4.122477] Detected PIPT I-cache on CPU46
> [ 4.122503] GICv3: CPU46: found redistributor 50302 region
> 2:0x000004004d480000
> [ 4.122560] CPU46: using LPI pending table @0x0000001fb5100000
> [ 4.122668] CPU46: Booted secondary processor 0x0000050302 [0x410fd082]
> [ 4.205126] Detected PIPT I-cache on CPU47
> [ 4.205152] GICv3: CPU47: found redistributor 50303 region
> 2:0x000004004d4c0000
> [ 4.205209] CPU47: using LPI pending table @0x0000001fb5110000
> [ 4.205316] CPU47: Booted secondary processor 0x0000050303 [0x410fd082]
> [ 4.279425] Detected PIPT I-cache on CPU48
> [ 4.279458] GICv3: CPU48: found redistributor 70000 region
> 3:0x000004006d100000
> [ 4.279519] CPU48: using LPI pending table @0x0000001fb51b0000
> [ 4.279669] CPU48: Booted secondary processor 0x0000070000 [0x410fd082]
> [ 4.362072] Detected PIPT I-cache on CPU49
> [ 4.362101] GICv3: CPU49: found redistributor 70001 region
> 3:0x000004006d140000
> [ 4.362163] CPU49: using LPI pending table @0x0000001fb5250000
> [ 4.362305] CPU49: Booted secondary processor 0x0000070001 [0x410fd082]
> [ 4.444725] Detected PIPT I-cache on CPU50
> [ 4.444753] GICv3: CPU50: found redistributor 70002 region
> 3:0x000004006d180000
> [ 4.444817] CPU50: using LPI pending table @0x0000001fb5290000
> [ 4.444962] CPU50: Booted secondary processor 0x0000070002 [0x410fd082]
> [ 4.527375] Detected PIPT I-cache on CPU51
> [ 4.527404] GICv3: CPU51: found redistributor 70003 region
> 3:0x000004006d1c0000
> [ 4.527466] CPU51: using LPI pending table @0x0000001fb5310000
> [ 4.527609] CPU51: Booted secondary processor 0x0000070003 [0x410fd082]
> [ 4.610026] Detected PIPT I-cache on CPU52
> [ 4.610058] GICv3: CPU52: found redistributor 70100 region
> 3:0x000004006d200000
> [ 4.610122] CPU52: using LPI pending table @0x0000001fb5380000
> [ 4.610273] CPU52: Booted secondary processor 0x0000070100 [0x410fd082]
> [ 4.692677] Detected PIPT I-cache on CPU53
> [ 4.692706] GICv3: CPU53: found redistributor 70101 region
> 3:0x000004006d240000
> [ 4.692769] CPU53: using LPI pending table @0x0000001fb53d0000
> [ 4.692912] CPU53: Booted secondary processor 0x0000070101 [0x410fd082]
> [ 4.775327] Detected PIPT I-cache on CPU54
> [ 4.775356] GICv3: CPU54: found redistributor 70102 region
> 3:0x000004006d280000
> [ 4.775417] CPU54: using LPI pending table @0x0000001fb4c50000
> [ 4.775561] CPU54: Booted secondary processor 0x0000070102 [0x410fd082]
> [ 4.857978] Detected PIPT I-cache on CPU55
> [ 4.858007] GICv3: CPU55: found redistributor 70103 region
> 3:0x000004006d2c0000
> [ 4.858069] CPU55: using LPI pending table @0x0000001fb4cc0000
> [ 4.858214] CPU55: Booted secondary processor 0x0000070103 [0x410fd082]
> [ 4.940631] Detected PIPT I-cache on CPU56
> [ 4.940663] GICv3: CPU56: found redistributor 70200 region
> 3:0x000004006d300000
> [ 4.940726] CPU56: using LPI pending table @0x0000001fb4d10000
> [ 4.940880] CPU56: Booted secondary processor 0x0000070200 [0x410fd082]
> [ 5.023280] Detected PIPT I-cache on CPU57
> [ 5.023309] GICv3: CPU57: found redistributor 70201 region
> 3:0x000004006d340000
> [ 5.023372] CPU57: using LPI pending table @0x0000001fb4d80000
> [ 5.023516] CPU57: Booted secondary processor 0x0000070201 [0x410fd082]
> [ 5.105930] Detected PIPT I-cache on CPU58
> [ 5.105959] GICv3: CPU58: found redistributor 70202 region
> 3:0x000004006d380000
> [ 5.106021] CPU58: using LPI pending table @0x0000001fb4dc0000
> [ 5.106166] CPU58: Booted secondary processor 0x0000070202 [0x410fd082]
> [ 5.188582] Detected PIPT I-cache on CPU59
> [ 5.188611] GICv3: CPU59: found redistributor 70203 region
> 3:0x000004006d3c0000
> [ 5.188672] CPU59: using LPI pending table @0x0000001fb4e50000
> [ 5.188815] CPU59: Booted secondary processor 0x0000070203 [0x410fd082]
> [ 5.271235] Detected PIPT I-cache on CPU60
> [ 5.271267] GICv3: CPU60: found redistributor 70300 region
> 3:0x000004006d400000
> [ 5.271330] CPU60: using LPI pending table @0x0000001fb4ed0000
> [ 5.271481] CPU60: Booted secondary processor 0x0000070300 [0x410fd082]
> [ 5.353884] Detected PIPT I-cache on CPU61
> [ 5.353913] GICv3: CPU61: found redistributor 70301 region
> 3:0x000004006d440000
> [ 5.353979] CPU61: using LPI pending table @0x0000001fb4f00000
> [ 5.354121] CPU61: Booted secondary processor 0x0000070301 [0x410fd082]
> [ 5.436534] Detected PIPT I-cache on CPU62
> [ 5.436563] GICv3: CPU62: found redistributor 70302 region
> 3:0x000004006d480000
> [ 5.436626] CPU62: using LPI pending table @0x0000001fb4f90000
> [ 5.436769] CPU62: Booted secondary processor 0x0000070302 [0x410fd082]
> [ 5.519187] Detected PIPT I-cache on CPU63
> [ 5.519217] GICv3: CPU63: found redistributor 70303 region
> 3:0x000004006d4c0000
> [ 5.519280] CPU63: using LPI pending table @0x0000001fb4fc0000
> [ 5.519421] CPU63: Booted secondary processor 0x0000070303 [0x410fd082]
> [ 5.519506] smp: Brought up 4 nodes, 64 CPUs
> [ 7.040874] SMP: Total of 64 processors activated.
> [ 7.045720] CPU features: detected: GIC system register CPU interface
> [ 7.052240] CPU features: detected: 32-bit EL0 Support
> [ 7.144026] CPU: All CPU(s) started at EL2
> [ 7.148298] alternatives: patching kernel code
> [ 7.155277] Unable to handle kernel NULL pointer dereference at
> virtual address 0000000000000000
> [ 7.164163] Mem abort info:
> [ 7.166978] ESR = 0x96000004
> [ 7.170061] Exception class = DABT (current EL), IL = 32 bits
> [ 7.176043] SET = 0, FnV = 0
> [ 7.179121] EA = 0, S1PTW = 0
> [ 7.182291] Data abort info:
> [ 7.185193] ISV = 0, ISS = 0x00000004
> [ 7.189066] CM = 0, WnR = 0
> [ 7.192056] [0000000000000000] user address but active_mm is swapper
> [ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
> [ 7.204107] Modules linked in:
> [ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
> 4.19.0-00002-g3ed52fd-dirty #807
> [ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
> [ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
> [ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
> [ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
> [ 7.237797] sp : ffff00000947bc60
> [ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
> [ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
> [ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
> [ 7.257235] x23: 0000000000000010 x22: 0000000000000001
> [ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
> [ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
> [ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
> [ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
> [ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
> [ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
> [ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
> [ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
> [ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
> [ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
> [ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
> [ 7.321613] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
> [ 7.328382] Call trace:
> [ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
> [ 7.335420] destroy_sched_domain+0x20/0x70
> [ 7.339642] cpu_attach_domain+0xc8/0x2e8
> [ 7.343687] build_sched_domains+0xd44/0xde0
> [ 7.347996] sched_init_domains+0x68/0x88
> [ 7.352044] sched_init_smp+0x2c/0x7c
> [ 7.355738] kernel_init_freeable+0xdc/0x244
> [ 7.360049] kernel_init+0x10/0x108
> [ 7.363567] ret_from_fork+0x10/0x18
> [ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
> [ 7.373400] ---[ end trace 8150af869a14363f ]---
> [ 7.378076] Kernel panic - not syncing: Attempted to kill init!
> exitcode=0x0000000b
> [ 7.378076]
> [ 7.387320] SMP: stopping secondary CPUs
> [ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill
> init! exitcode=0x0000000b
> [ 7.391297] ]---
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-10-23 10:30 Crash report: Broken NUMA distance map causes crash on arm64 system John Garry
2018-10-25 11:01 ` John Garry
@ 2018-10-30 9:26 ` Peter Zijlstra
2018-10-30 9:55 ` John Garry
1 sibling, 1 reply; 13+ messages in thread
From: Peter Zijlstra @ 2018-10-30 9:26 UTC (permalink / raw)
To: John Garry
Cc: linux-arm-kernel, linux-kernel, Will Deacon, Catalin Marinas,
Rob Herring, devicetree, Frank Rowand, Ingo Molnar, Linuxarm
On Tue, Oct 23, 2018 at 11:30:11AM +0100, John Garry wrote:
> Hi all,
>
> I have stumbled upon this crash on my arm64 system:
>
> [ 7.040874] SMP: Total of 64 processors activated.
> [ 7.045720] CPU features: detected: GIC system register CPU interface
> [ 7.052240] CPU features: detected: 32-bit EL0 Support
> [ 7.144026] CPU: All CPU(s) started at EL2
> [ 7.148298] alternatives: patching kernel code
> [ 7.155277] Unable to handle kernel NULL pointer dereference at virtual
> address 0000000000000000
> [ 7.164163] Mem abort info:
> [ 7.166978] ESR = 0x96000004
> [ 7.170061] Exception class = DABT (current EL), IL = 32 bits
> [ 7.176043] SET = 0, FnV = 0
> [ 7.179121] EA = 0, S1PTW = 0
> [ 7.182291] Data abort info:
> [ 7.185193] ISV = 0, ISS = 0x00000004
> [ 7.189066] CM = 0, WnR = 0
> [ 7.192056] [0000000000000000] user address but active_mm is swapper
> [ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
> [ 7.204107] Modules linked in:
> [ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
> 4.19.0-00002-g3ed52fd-dirty #807
> [ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
> [ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
> [ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
> [ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
> [ 7.237797] sp : ffff00000947bc60
> [ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
> [ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
> [ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
> [ 7.257235] x23: 0000000000000010 x22: 0000000000000001
> [ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
> [ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
> [ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
> [ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
> [ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
> [ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
> [ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
> [ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
> [ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
> [ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
> [ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
> [ 7.321613] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
> [ 7.328382] Call trace:
> [ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
> [ 7.335420] destroy_sched_domain+0x20/0x70
> [ 7.339642] cpu_attach_domain+0xc8/0x2e8
> [ 7.343687] build_sched_domains+0xd44/0xde0
> [ 7.347996] sched_init_domains+0x68/0x88
> [ 7.352044] sched_init_smp+0x2c/0x7c
> [ 7.355738] kernel_init_freeable+0xdc/0x244
> [ 7.360049] kernel_init+0x10/0x108
> [ 7.363567] ret_from_fork+0x10/0x18
> [ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
> [ 7.373400] ---[ end trace 8150af869a14363f ]---
> [ 7.378076] Kernel panic - not syncing: Attempted to kill init!
> exitcode=0x0000000b
> [ 7.378076]
> [ 7.387320] SMP: stopping secondary CPUs
> [ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill init!
> exitcode=0x0000000b
> [ 7.391297] ]---
>
> I will straightaway note that I have been fiddling with my board's device
> tree, specifically (breaking!) the NUMA distance map, like this:
>
> distance-matrix = <0 0 10>,
> <0 1 15>,
> <0 2 20>,
> <0 3 25>,
> <1 0 15>,
> <1 1 10>,
> <1 2 25>,
> <1 3 30>,
> <2 0 20>,
> <2 1 25>,
> <2 2 10>,
> <2 3 15>,
> <3 0 10>,* should be same as 0->3 and > 10
> <3 1 10>,* should be same as 1->3 and > 10
> <3 2 15>,
> <3 3 15>;* should be 10
>
> However I don't think that this should crash the kernel. I'd say
> of_numa_parse_distance_map_v1() should robustly handle broken maps, probably
> by erroring and causing NUMA to be disabled.
If build with CONFIG_SCHED_DEBUG=y and boot with "sched_debug" I think
sched_init_numa() should yell at you for the above 'mistake'.
> However I am interested to know if the scheduler crash is a real problem.
Ideally we'd not crash of course.. let me see if I can still make sense
of that topology code.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-10-30 9:26 ` Peter Zijlstra
@ 2018-10-30 9:55 ` John Garry
2018-10-30 15:35 ` John Garry
0 siblings, 1 reply; 13+ messages in thread
From: John Garry @ 2018-10-30 9:55 UTC (permalink / raw)
To: Peter Zijlstra
Cc: linux-arm-kernel, linux-kernel, Will Deacon, Catalin Marinas,
Rob Herring, devicetree, Frank Rowand, Ingo Molnar, Linuxarm,
Anshuman Khandual
On 30/10/2018 09:26, Peter Zijlstra wrote:
> On Tue, Oct 23, 2018 at 11:30:11AM +0100, John Garry wrote:
>> Hi all,
>>
>> I have stumbled upon this crash on my arm64 system:
>>
>> [ 7.040874] SMP: Total of 64 processors activated.
>> [ 7.045720] CPU features: detected: GIC system register CPU interface
>> [ 7.052240] CPU features: detected: 32-bit EL0 Support
>> [ 7.144026] CPU: All CPU(s) started at EL2
>> [ 7.148298] alternatives: patching kernel code
>> [ 7.155277] Unable to handle kernel NULL pointer dereference at virtual
>> address 0000000000000000
>> [ 7.164163] Mem abort info:
>> [ 7.166978] ESR = 0x96000004
>> [ 7.170061] Exception class = DABT (current EL), IL = 32 bits
>> [ 7.176043] SET = 0, FnV = 0
>> [ 7.179121] EA = 0, S1PTW = 0
>> [ 7.182291] Data abort info:
>> [ 7.185193] ISV = 0, ISS = 0x00000004
>> [ 7.189066] CM = 0, WnR = 0
>> [ 7.192056] [0000000000000000] user address but active_mm is swapper
>> [ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
>> [ 7.204107] Modules linked in:
>> [ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
>> 4.19.0-00002-g3ed52fd-dirty #807
>> [ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
>> [ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
>> [ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
>> [ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
>> [ 7.237797] sp : ffff00000947bc60
>> [ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
>> [ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
>> [ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
>> [ 7.257235] x23: 0000000000000010 x22: 0000000000000001
>> [ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
>> [ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
>> [ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
>> [ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
>> [ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
>> [ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
>> [ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
>> [ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
>> [ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
>> [ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
>> [ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
>> [ 7.321613] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
>> [ 7.328382] Call trace:
>> [ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
>> [ 7.335420] destroy_sched_domain+0x20/0x70
>> [ 7.339642] cpu_attach_domain+0xc8/0x2e8
>> [ 7.343687] build_sched_domains+0xd44/0xde0
>> [ 7.347996] sched_init_domains+0x68/0x88
>> [ 7.352044] sched_init_smp+0x2c/0x7c
>> [ 7.355738] kernel_init_freeable+0xdc/0x244
>> [ 7.360049] kernel_init+0x10/0x108
>> [ 7.363567] ret_from_fork+0x10/0x18
>> [ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
>> [ 7.373400] ---[ end trace 8150af869a14363f ]---
>> [ 7.378076] Kernel panic - not syncing: Attempted to kill init!
>> exitcode=0x0000000b
>> [ 7.378076]
>> [ 7.387320] SMP: stopping secondary CPUs
>> [ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill init!
>> exitcode=0x0000000b
>> [ 7.391297] ]---
>>
>> I will straightaway note that I have been fiddling with my board's device
>> tree, specifically (breaking!) the NUMA distance map, like this:
>>
>> distance-matrix = <0 0 10>,
>> <0 1 15>,
>> <0 2 20>,
>> <0 3 25>,
>> <1 0 15>,
>> <1 1 10>,
>> <1 2 25>,
>> <1 3 30>,
>> <2 0 20>,
>> <2 1 25>,
>> <2 2 10>,
>> <2 3 15>,
>> <3 0 10>,* should be same as 0->3 and > 10
>> <3 1 10>,* should be same as 1->3 and > 10
>> <3 2 15>,
>> <3 3 15>;* should be 10
>>
>> However I don't think that this should crash the kernel. I'd say
>> of_numa_parse_distance_map_v1() should robustly handle broken maps, probably
>> by erroring and causing NUMA to be disabled.
>
> If build with CONFIG_SCHED_DEBUG=y and boot with "sched_debug" I think
> sched_init_numa() should yell at you for the above 'mistake'.
>
Right, I have since turned on debug and it was complaining - I'll
provide a log.
>> However I am interested to know if the scheduler crash is a real problem.
>
> Ideally we'd not crash of course.. let me see if I can still make sense
> of that topology code.
JFYI, https://lkml.org/lkml/2018/10/26/527
So it seems to be that setting the distance between separate nodes to
LOCAL_DISTANCE causes the problem. x86 NUMA code - like arm64 - allows
this. However ACPI SLIT validation rejects this, unlike OF distance map
parsing.
Thanks,
John
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-10-30 9:55 ` John Garry
@ 2018-10-30 15:35 ` John Garry
2018-10-31 20:46 ` Peter Zijlstra
0 siblings, 1 reply; 13+ messages in thread
From: John Garry @ 2018-10-30 15:35 UTC (permalink / raw)
To: Peter Zijlstra
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
linux-kernel, Linuxarm, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On 30/10/2018 09:55, John Garry wrote:
> On 30/10/2018 09:26, Peter Zijlstra wrote:
>> On Tue, Oct 23, 2018 at 11:30:11AM +0100, John Garry wrote:
>>> Hi all,
>>>
>>> I have stumbled upon this crash on my arm64 system:
>>>
>>> [ 7.040874] SMP: Total of 64 processors activated.
>>> [ 7.045720] CPU features: detected: GIC system register CPU interface
>>> [ 7.052240] CPU features: detected: 32-bit EL0 Support
>>> [ 7.144026] CPU: All CPU(s) started at EL2
>>> [ 7.148298] alternatives: patching kernel code
>>> [ 7.155277] Unable to handle kernel NULL pointer dereference at
>>> virtual
>>> address 0000000000000000
>>> [ 7.164163] Mem abort info:
>>> [ 7.166978] ESR = 0x96000004
>>> [ 7.170061] Exception class = DABT (current EL), IL = 32 bits
>>> [ 7.176043] SET = 0, FnV = 0
>>> [ 7.179121] EA = 0, S1PTW = 0
>>> [ 7.182291] Data abort info:
>>> [ 7.185193] ISV = 0, ISS = 0x00000004
>>> [ 7.189066] CM = 0, WnR = 0
>>> [ 7.192056] [0000000000000000] user address but active_mm is swapper
>>> [ 7.198480] Internal error: Oops: 96000004 [#1] PREEMPT SMP
>>> [ 7.204107] Modules linked in:
>>> [ 7.207189] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
>>> 4.19.0-00002-g3ed52fd-dirty #807
>>> [ 7.216596] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
>>> [ 7.223102] pstate: 20000005 (nzCv daif -PAN -UAO)
>>> [ 7.227946] pc : __ll_sc_atomic_sub_return+0x4/0x20
>>> [ 7.232873] lr : free_sched_groups.part.1+0x40/0x98
>>> [ 7.237797] sp : ffff00000947bc60
>>> [ 7.241138] x29: ffff00000947bc60 x28: ffff801fb49ba000
>>> [ 7.246504] x27: ffff000009109b38 x26: ffff801fb4a38000
>>> [ 7.251869] x25: ffff801fb48cc000 x24: ffff000009109000
>>> [ 7.257235] x23: 0000000000000010 x22: 0000000000000001
>>> [ 7.262599] x21: ffff801fb4a0b000 x20: ffff801fb4a65f00
>>> [ 7.267964] x19: ffff801fb4a65f80 x18: 0000000000000400
>>> [ 7.273329] x17: 00000000ffffffff x16: 0000000000000000
>>> [ 7.278694] x15: ffff801ffbffef80 x14: ffff7e007ed14bc0
>>> [ 7.284059] x13: 00000000000000c0 x12: 000000000000003f
>>> [ 7.289423] x11: ffff801fb62258b8 x10: 0000000000000001
>>> [ 7.294788] x9 : ffff801fb44b0100 x8 : 000000000000000f
>>> [ 7.300153] x7 : ffff801fb48b1900 x6 : 0000801ff2efb000
>>> [ 7.305518] x5 : ffff0000091096f0 x4 : 0000000000000400
>>> [ 7.310882] x3 : ffff801fb4a65000 x2 : ffff801fb4a0b000
>>> [ 7.316247] x1 : 0000000000000000 x0 : 0000000000000001
>>> [ 7.321613] Process swapper/0 (pid: 1, stack limit =
>>> 0x(____ptrval____))
>>> [ 7.328382] Call trace:
>>> [ 7.330847] __ll_sc_atomic_sub_return+0x4/0x20
>>> [ 7.335420] destroy_sched_domain+0x20/0x70
>>> [ 7.339642] cpu_attach_domain+0xc8/0x2e8
>>> [ 7.343687] build_sched_domains+0xd44/0xde0
>>> [ 7.347996] sched_init_domains+0x68/0x88
>>> [ 7.352044] sched_init_smp+0x2c/0x7c
>>> [ 7.355738] kernel_init_freeable+0xdc/0x244
>>> [ 7.360049] kernel_init+0x10/0x108
>>> [ 7.363567] ret_from_fork+0x10/0x18
>>> [ 7.367174] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
>>> [ 7.373400] ---[ end trace 8150af869a14363f ]---
>>> [ 7.378076] Kernel panic - not syncing: Attempted to kill init!
>>> exitcode=0x0000000b
>>> [ 7.378076]
>>> [ 7.387320] SMP: stopping secondary CPUs
>>> [ 7.391297] ---[ end Kernel panic - not syncing: Attempted to kill
>>> init!
>>> exitcode=0x0000000b
>>> [ 7.391297] ]---
>>>
>>> I will straightaway note that I have been fiddling with my board's
>>> device
>>> tree, specifically (breaking!) the NUMA distance map, like this:
>>>
>>> distance-matrix = <0 0 10>,
>>> <0 1 15>,
>>> <0 2 20>,
>>> <0 3 25>,
>>> <1 0 15>,
>>> <1 1 10>,
>>> <1 2 25>,
>>> <1 3 30>,
>>> <2 0 20>,
>>> <2 1 25>,
>>> <2 2 10>,
>>> <2 3 15>,
>>> <3 0 10>,* should be same as 0->3 and > 10
>>> <3 1 10>,* should be same as 1->3 and > 10
>>> <3 2 15>,
>>> <3 3 15>;* should be 10
>>>
>>> However I don't think that this should crash the kernel. I'd say
>>> of_numa_parse_distance_map_v1() should robustly handle broken maps,
>>> probably
>>> by erroring and causing NUMA to be disabled.
>>
>> If build with CONFIG_SCHED_DEBUG=y and boot with "sched_debug" I think
>> sched_init_numa() should yell at you for the above 'mistake'.
>>
>
> Right, I have since turned on debug and it was complaining - I'll
> provide a log.
As promised:
EFI stub: Booting Linux Kernel...
SmiGraphicsOutputQueryMode +
SmiGraphicsOutputQueryMode -
SmiGraphicsOutputQueryMode +
SmiGraphicsOutputQueryMode -
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...
[SAL_ClearAffiliationSMP,325]it is going to hard reset
dev:0x500E004AAAAAAA1F
SAS ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
GMAC ExitBootServicesEvent
OHCI ExitBootServicesEvent
IPMI ExitBootService Event
TransferSmbiosToBMC EVENT.
Transfer Smbios Table To iBMC Success.
GetVariable Status : Not Found.
[ 0.000000] Booting Linux on physical CPU 0x0000010000 [0x410fd082]
[ 0.000000] Linux version 4.19.0-rc6-00002-gea67197-dirty
(johnpgarry@johnpgarry-ThinkCentre-M93p) (gcc version 7.3.1 20180425
[linaro-7.3-2018.05-rc1 revision
38aec9a676236eaa42ca03ccb3a6c1dd0182c29f] (Linaro GCC 7.3-2018.05-rc1))
#877 SMP PREEMPT Tue Oct 30 15:10:30 GMT 2018
[ 0.000000] Machine model: Hisilicon Hip07 D05 Development Board
[ 0.000000] earlycon: pl11 at MMIO 0x00000000602b0000 (options '')
[ 0.000000] bootconsole [pl11] enabled
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: EFI v2.60 by EDK II
[ 0.000000] efi: SMBIOS=0x3f050000 SMBIOS 3.0=0x39b00000
ACPI=0x39bd0000 ACPI 2.0=0x39bd0014 MEMATTR=0x3cc99018
[ 0.000000] crashkernel reserved: 0x0000000002000000 -
0x0000000012000000 (256 MB)
[ 0.000000] cma: Reserved 32 MiB at 0x000000003d000000
[ 0.000000] earlycon: pl11 at MMIO32 0x00000000602b0000 (options
'115200n8')
[ 0.000000] ------------[ cut here ]------------
[ 0.000000] console 'pl11' already registered
[ 0.000000] WARNING: CPU: 0 PID: 0 at kernel/printk/printk.c:2586
register_console+0x58/0x3c8
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted
4.19.0-rc6-00002-gea67197-dirty #877
[ 0.000000] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 0.000000] pstate: 60000085 (nZCv daIf -PAN -UAO)
[ 0.000000] pc : register_console+0x58/0x3c8
[ 0.000000] lr : register_console+0x58/0x3c8
[ 0.000000] sp : ffff000009113de0
[ 0.000000] x29: ffff000009113de0 x28: 0000000000fc0018
[ 0.000000] x27: 000000003cc81018 x26: ffff0000090e2098
[ 0.000000] x25: 0000000000004b10 x24: ffff0000090f5348
[ 0.000000] x23: ffff0000090e2098 x22: ffff000009274000
[ 0.000000] x21: ffff00000920e6d0 x20: 0000000000004b10
[ 0.000000] x19: ffff00000920e6d0 x18: ffffffffffffffff
[ 0.000000] x17: 0000000000000000 x16: 0000000000000000
[ 0.000000] x15: ffff0000091196c8 x14: ffff0000892746ef
[ 0.000000] x13: ffff0000092746fd x12: ffff000009119940
[ 0.000000] x11: ffff0000085f42b8 x10: ffff000009113ab0
[ 0.000000] x9 : 000000000000000d x8 : 722079646165726c
[ 0.000000] x7 : 61202731316c7027 x6 : 000000000000000d
[ 0.000000] x5 : 0000000000000000 x4 : 0000000000000000
[ 0.000000] x3 : ffffffffffffffff x2 : ffff000009131640
[ 0.000000] x1 : 0000000000000000 x0 : 0000000000000000
[ 0.000000] Call trace:
[ 0.000000] register_console+0x58/0x3c8
[ 0.000000] of_setup_earlycon+0x20c/0x22c
[ 0.000000] early_init_dt_scan_chosen_stdout+0x158/0x19c
[ 0.000000] acpi_boot_table_init+0x150/0x184
[ 0.000000] setup_arch+0x1cc/0x514
[ 0.000000] start_kernel+0x80/0x418
[ 0.000000] random: get_random_bytes called from
print_oops_end_marker+0x48/0x68 with crng_init=0
[ 0.000000] ---[ end trace 0000000000000000 ]---
[ 0.000000] OF: NUMA: parsing numa-distance-map-v1
[ 0.000000] NUMA: Warning: invalid distance parameter, from=3 to=3
distance=15
[ 0.000000] NUMA: NODE_DATA [mem 0x1ffbffe840-0x1ffbffffff]
[ 0.000000] NUMA: Initmem setup node 1 [<memory-less node>]
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf41fc0-0x41ffbf4377f]
[ 0.000000] NUMA: NODE_DATA(1) on node 2
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf40800-0x41ffbf41fbf]
[ 0.000000] NUMA: Initmem setup node 3 [<memory-less node>]
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbf3f040-0x41ffbf407ff]
[ 0.000000] NUMA: NODE_DATA(3) on node 2
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000000000000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x0000041ffbffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x0000000031902fff]
[ 0.000000] node 0: [mem 0x0000000031903000-0x0000000031906fff]
[ 0.000000] node 0: [mem 0x0000000031907000-0x000000003999ffff]
[ 0.000000] node 0: [mem 0x00000000399a0000-0x0000000039a3ffff]
[ 0.000000] node 0: [mem 0x0000000039a40000-0x0000000039a8ffff]
[ 0.000000] node 0: [mem 0x0000000039a90000-0x0000000039a9ffff]
[ 0.000000] node 0: [mem 0x0000000039aa0000-0x0000000039aeffff]
[ 0.000000] node 0: [mem 0x0000000039af0000-0x0000000039bbffff]
[ 0.000000] node 0: [mem 0x0000000039bc0000-0x0000000039bdffff]
[ 0.000000] node 0: [mem 0x0000000039be0000-0x000000003a05ffff]
[ 0.000000] node 0: [mem 0x000000003a060000-0x000000003f04ffff]
[ 0.000000] node 0: [mem 0x000000003f050000-0x000000003f07ffff]
[ 0.000000] node 0: [mem 0x000000003f080000-0x000000003fbfffff]
[ 0.000000] node 0: [mem 0x0000001040000000-0x0000001ffbffffff]
[ 0.000000] node 2: [mem 0x0000041000000000-0x0000041ffbffffff]
[ 0.000000] Initmem setup node 0 [mem
0x0000000000000000-0x0000001ffbffffff]
[ 0.000000] Could not find start_pfn for node 1
[ 0.000000] Initmem setup node 1 [mem
0x0000000000000000-0x0000000000000000]
[ 0.000000] Initmem setup node 2 [mem
0x0000041000000000-0x0000041ffbffffff]
[ 0.000000] Could not find start_pfn for node 3
[ 0.000000] Initmem setup node 3 [mem
0x0000000000000000-0x0000000000000000]
[ 0.000000] percpu: Embedded 23 pages/cpu @(____ptrval____) s56408
r8192 d29608 u94208
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for EL2 vector hardening
[ 0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[ 0.000000] Built 4 zonelists, mobility grouping on. Total pages:
32996880
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: BOOT_IMAGE=/john/Image rdinit=/init
crashkernel=256M@32M console=ttyAMA0,115200
earlycon=pl011,mmio,0x602b0000 acpi=off pcie_aspm=off earlycon sched_debug
[ 0.000000] log_buf_len individual max cpu contribution: 4096 bytes
[ 0.000000] log_buf_len total cpu_extra contributions: 258048 bytes
[ 0.000000] log_buf_len min size: 131072 bytes
[ 0.000000] log_buf_len: 524288 bytes
[ 0.000000] early log buf free: 123328(94%)
[ 0.000000] software IO TLB: mapped [mem 0x359a0000-0x399a0000] (64MB)
[ 0.000000] Memory: 131332180K/134082560K available (10684K kernel
code, 1374K rwdata, 4896K rodata, 1344K init, 384K bss, 2717612K
reserved, 32768K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=64, Nodes=4
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv3: Distributor has no Range Selector support
[ 0.000000] GICv3: VLPI support, direct LPI support
[ 0.000000] ITS [mem 0x4c000000-0x4c03ffff]
[ 0.000000] ITS@0x000000004c000000: Using ITS number 0
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000000004c000000: allocated 524288 Devices
@1fbb000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000000004c000000: allocated 65536 Virtual CPUs
@1fbb980000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000000004c000000: allocated 512 Interrupt
Collections @1fbb929000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x6c000000-0x6c03ffff]
[ 0.000000] ITS@0x000000006c000000: Using ITS number 1
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000000006c000000: allocated 524288 Devices
@1fbac00000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000000006c000000: allocated 65536 Virtual CPUs
@1fbba00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000000006c000000: allocated 512 Interrupt
Collections @1fbb92a000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0xc6000000-0xc603ffff]
[ 0.000000] ITS@0x00000000c6000000: Using ITS number 2
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000000c6000000: allocated 524288 Devices
@1fba800000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000000c6000000: allocated 65536 Virtual CPUs
@1fbba80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt
Collections @1fbb92c000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x8c6000000-0x8c603ffff]
[ 0.000000] ITS@0x00000008c6000000: Using ITS number 3
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000008c6000000: allocated 524288 Devices
@1fba400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000008c6000000: allocated 65536 Virtual CPUs
@1fbbb00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000008c6000000: allocated 512 Interrupt
Collections @1fbb92e000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x4004c000000-0x4004c03ffff]
[ 0.000000] ITS@0x000004004c000000: Using ITS number 4
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000004004c000000: allocated 524288 Devices
@1fba000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000004004c000000: allocated 65536 Virtual CPUs
@1fbbb80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000004004c000000: allocated 512 Interrupt
Collections @1fbb92f000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x4006c000000-0x4006c03ffff]
[ 0.000000] ITS@0x000004006c000000: Using ITS number 5
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000004006c000000: allocated 524288 Devices
@1fb9800000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000004006c000000: allocated 65536 Virtual CPUs
@1fb9c80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000004006c000000: allocated 512 Interrupt
Collections @1fb9c10000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x400c6000000-0x400c603ffff]
[ 0.000000] ITS@0x00000400c6000000: Using ITS number 6
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000400c6000000: allocated 524288 Devices
@1fb9400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000400c6000000: allocated 65536 Virtual CPUs
@1fb9d00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000400c6000000: allocated 512 Interrupt
Collections @1fb9c11000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x408c6000000-0x408c603ffff]
[ 0.000000] ITS@0x00000408c6000000: Using ITS number 7
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000408c6000000: allocated 524288 Devices
@1fb9000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000408c6000000: allocated 65536 Virtual CPUs
@1fb9d80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000408c6000000: allocated 512 Interrupt
Collections @1fb9c13000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] GIC: using LPI property table @0x0000001fb9c40000
[ 0.000000] ITS: Using DirectLPI for VPE invalidation
[ 0.000000] ITS: Enabling GICv4 support
[ 0.000000] GICv3: CPU0: found redistributor 10000 region
0:0x000000004d100000
[ 0.000000] CPU0: using LPI pending table @0x0000001fb9c50000
[ 0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff
max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[ 0.000001] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps
every 4398046511100ns
[ 0.008253] Console: colour dummy device 80x25
[ 0.012797] mempolicy: Enabling automatic NUMA balancing. Configure
with numa_balancing= or the kernel.numa_balancing sysctl
[ 0.024161] Calibrating delay loop (skipped), value calculated using
timer frequency.. 100.00 BogoMIPS (lpj=200000)
[ 0.034714] pid_max: default: 65536 minimum: 512
[ 0.039437] Security Framework initialized
[ 0.066552] Dentry cache hash table entries: 8388608 (order: 14,
67108864 bytes)
[ 0.085423] Inode-cache hash table entries: 4194304 (order: 13,
33554432 bytes)
[ 0.093195] Mount-cache hash table entries: 131072 (order: 8, 1048576
bytes)
[ 0.100677] Mountpoint-cache hash table entries: 131072 (order: 8,
1048576 bytes)
[ 0.152289] ASID allocator initialised with 32768 entries
[ 0.157776] rcu: Hierarchical SRCU implementation.
[ 0.170653] Platform MSI: interrupt-controller@4c000000 domain created
[ 0.177254] Platform MSI: interrupt-controller@6c000000 domain created
[ 0.183853] Platform MSI: interrupt-controller@c6000000 domain created
[ 0.190452] Platform MSI: interrupt-controller@8,c6000000 domain created
[ 0.197227] Platform MSI: interrupt-controller@400,4c000000 domain
created
[ 0.204178] Platform MSI: interrupt-controller@400,6c000000 domain
created
[ 0.211129] Platform MSI: interrupt-controller@400,c6000000 domain
created
[ 0.218079] Platform MSI: interrupt-controller@408,c6000000 domain
created
[ 0.225076] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@4c000000 domain created
[ 0.233960] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@6c000000 domain created
[ 0.242844] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@c6000000 domain created
[ 0.251729] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@8,c6000000 domain
created
[ 0.260790] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,4c000000 domain
created
[ 0.270026] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,6c000000 domain
created
[ 0.279261] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,c6000000 domain
created
[ 0.288497] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@408,c6000000 domain
created
[ 0.298192] Remapping and enabling EFI services.
[ 0.310865] smp: Bringing up secondary CPUs ...
[ 0.398048] Detected PIPT I-cache on CPU1
[ 0.398060] GICv3: CPU1: found redistributor 10001 region
0:0x000000004d140000
[ 0.398118] CPU1: using LPI pending table @0x0000041fb81e0000
[ 0.398238] CPU1: Booted secondary processor 0x0000010001 [0x410fd082]
[ 0.480668] Detected PIPT I-cache on CPU2
[ 0.480674] GICv3: CPU2: found redistributor 10002 region
0:0x000000004d180000
[ 0.480730] CPU2: using LPI pending table @0x0000041fb81f0000
[ 0.480833] CPU2: Booted secondary processor 0x0000010002 [0x410fd082]
[ 0.563294] Detected PIPT I-cache on CPU3
[ 0.563301] GICv3: CPU3: found redistributor 10003 region
0:0x000000004d1c0000
[ 0.563360] CPU3: using LPI pending table @0x0000041fb8200000
[ 0.563466] CPU3: Booted secondary processor 0x0000010003 [0x410fd082]
[ 0.645921] Detected PIPT I-cache on CPU4
[ 0.645929] GICv3: CPU4: found redistributor 10100 region
0:0x000000004d200000
[ 0.645988] CPU4: using LPI pending table @0x0000041fb8210000
[ 0.646092] CPU4: Booted secondary processor 0x0000010100 [0x410fd082]
[ 0.728547] Detected PIPT I-cache on CPU5
[ 0.728554] GICv3: CPU5: found redistributor 10101 region
0:0x000000004d240000
[ 0.728611] CPU5: using LPI pending table @0x0000041fb8220000
[ 0.728714] CPU5: Booted secondary processor 0x0000010101 [0x410fd082]
[ 0.811174] Detected PIPT I-cache on CPU6
[ 0.811181] GICv3: CPU6: found redistributor 10102 region
0:0x000000004d280000
[ 0.811238] CPU6: using LPI pending table @0x0000041fb8230000
[ 0.811342] CPU6: Booted secondary processor 0x0000010102 [0x410fd082]
[ 0.893801] Detected PIPT I-cache on CPU7
[ 0.893808] GICv3: CPU7: found redistributor 10103 region
0:0x000000004d2c0000
[ 0.893864] CPU7: using LPI pending table @0x0000041fb8240000
[ 0.893969] CPU7: Booted secondary processor 0x0000010103 [0x410fd082]
[ 0.976430] Detected PIPT I-cache on CPU8
[ 0.976438] GICv3: CPU8: found redistributor 10200 region
0:0x000000004d300000
[ 0.976494] CPU8: using LPI pending table @0x0000041fb8250000
[ 0.976599] CPU8: Booted secondary processor 0x0000010200 [0x410fd082]
[ 1.059056] Detected PIPT I-cache on CPU9
[ 1.059063] GICv3: CPU9: found redistributor 10201 region
0:0x000000004d340000
[ 1.059118] CPU9: using LPI pending table @0x0000041fb8260000
[ 1.059221] CPU9: Booted secondary processor 0x0000010201 [0x410fd082]
[ 1.141682] Detected PIPT I-cache on CPU10
[ 1.141689] GICv3: CPU10: found redistributor 10202 region
0:0x000000004d380000
[ 1.141745] CPU10: using LPI pending table @0x0000041fb8270000
[ 1.141848] CPU10: Booted secondary processor 0x0000010202 [0x410fd082]
[ 1.224310] Detected PIPT I-cache on CPU11
[ 1.224317] GICv3: CPU11: found redistributor 10203 region
0:0x000000004d3c0000
[ 1.224375] CPU11: using LPI pending table @0x0000041fb8280000
[ 1.224477] CPU11: Booted secondary processor 0x0000010203 [0x410fd082]
[ 1.306938] Detected PIPT I-cache on CPU12
[ 1.306948] GICv3: CPU12: found redistributor 10300 region
0:0x000000004d400000
[ 1.307004] CPU12: using LPI pending table @0x0000041fb8290000
[ 1.307110] CPU12: Booted secondary processor 0x0000010300 [0x410fd082]
[ 1.389564] Detected PIPT I-cache on CPU13
[ 1.389571] GICv3: CPU13: found redistributor 10301 region
0:0x000000004d440000
[ 1.389629] CPU13: using LPI pending table @0x0000041fb82a0000
[ 1.389733] CPU13: Booted secondary processor 0x0000010301 [0x410fd082]
[ 1.472190] Detected PIPT I-cache on CPU14
[ 1.472198] GICv3: CPU14: found redistributor 10302 region
0:0x000000004d480000
[ 1.472252] CPU14: using LPI pending table @0x0000041fb82b0000
[ 1.472598] CPU14: Booted secondary processor 0x0000010302 [0x410fd082]
[ 1. 1.554883] CPU15: using LPI pending table @0x0000041fb82c0000
[ 1.554988] CPU15: Booted secondary processor 0x0000010303 [0x410fd082]
[ 1.637107] Detected PIPT I-cache on CPU16
[ 1.637119] GICv3: CPU16: found redistributor 30000 region
1:0x000000006d100000
[ 1.637182] CPU16: using LPI pending table @0x0000041fb82d0000
[ 1.637329] CPU16: Booted secondary processor 0x0000030000 [0x410fd082]
[ 1.719738] Detected PIPT I-cache on CPU17
[ 1.719747] GICv3: CPU17: found redistributor 30001 region
1:0x000000006d140000
[ 1.719806] CPU17: using LPI pending table @0x0000041fb82e0000
[ 1.719948] CPU17: Booted secondary processor 0x0000030001 [0x410fd082]
[ 1.802371] Detected PIPT I-cache on CPU18
[ 1.802380] GICv3: CPU18: found redistributor 30002 region
1:0x000000006d180000
[ 1.802441] CPU18: using LPI pending table @0x0000041fb82f0000
[ 1.802585] CPU18: Booted secondary processor 0x0000030002 [0x410fd082]
[ 1.885004] Detected PIPT I-cache on CPU19
[ 1.885013] GICv3: CPU19: found redistributor 30003 region
1:0x000000006d1c0000
[ 1.885074] CPU19: using LPI pending table @0x0000041fb8300000
[ 1.885215] CPU19: Booted secondary processor 0x0000030003 [0x410fd082]
[ 1.967639] Detected PIPT I-cache on CPU20
[ 1.967651] GICv3: CPU20: found redistributor 30100 region
1:0x000000006d200000
[ 1.967713] CPU20: using LPI pending table @0x0000041fb8310000
[ 1.967860] CPU20: Booted secondary processor 0x0000030100 [0x410fd082]
[ 2.050270] Detected PIPT I-cache on CPU21
[ 2.050279] GICv3: CPU21: found redistributor 30101 region
1:0x000000006d240000
[ 2.050341] CPU21: using LPI pending table @0x0000041fb8320000
[ 2.050483] CPU21: Booted secondary processor 0x0000030101 [0x410fd082]
[ 2.132903] Detected PIPT I-cache on CPU22
[ 2.132913] GICv3: CPU22: found redistributor 30102 region
1:0x000000006d280000
[ 2.132973] CPU22: using LPI pending table @0x0000041fb8330000
[ 2.133115] CPU22: Booted secondary processor 0x0000030102 [0x410fd082]
[ 2.215536] Detected PIPT I-cache on CPU23
[ 2.215546] GICv3: CPU23: found redistributor 30103 region
1:0x000000006d2c0000
[ 2.215606] CPU23: using LPI pending table @0x0000041fb8340000
[ 2.215749] CPU23: Booted secondary processor 0x0000030103 [0x410fd082]
[ 2.298171] Detected PIPT I-cache on CPU24
[ 2.298184] GICv3: CPU24: found redistributor 30200 region
1:0x000000006d300000
[ 2.298248] CPU24: using LPI pending table @0x0000041fb8350000
[ 2.298394] CPU24: Booted secondary processor 0x0000030200 [0x410fd082]
[ 2.380801] Detected PIPT I-cache on CPU25
[ 2.380811] GICv3: CPU25: found redistributor 30201 region
1:0x000000006d340000
[ 2.380872] CPU25: using LPI pending table @0x0000041fb8360000
[ 2.381013] CPU25: Booted secondary processor 0x0000030201 [0x410fd082]
[ 2.463433] Detected PIPT I-cache on CPU26
[ 2.463443] GICv3: CPU26: found redistributor 30202 region
1:0x000000006d380000
[ 2.463504] CPU26: using LPI pending table @0x0000041fb8370000
[ 2.463646] CPU26: Booted secondary processor 0x0000030202 [0x410fd082]
[ 2.546066] Detected PIPT I-cache on CPU27
[ 2.546076] GICv3: CPU27: found redistributor 30203 region
1:0x000000006d3c0000
[ 2.546139] CPU27: using LPI pending table @0x0000041fb8380000
[ 2.546282] CPU27: Booted secondary processor 0x0000030203 [0x410fd082]
[ 2.628701] Detected PIPT I-cache on CPU28
[ 2.628713] GICv3: CPU28: found redistributor 30300 region
1:0x000000006d400000
[ 2.628775] CPU28: using LPI pending table @0x0000041fb8390000
[ 2.628921] CPU28: Booted secondary processor 0x0000030300 [0x410fd082]
[ 2.711331] Detected PIPT I-cache on CPU29
[ 2.711342] GICv3: CPU29: found redistributor 30301 region
1:0x000000006d440000
[ 2.711403] CPU29: using LPI pending table @0x0000041fb83a0000
[ 2.711545] CPU29: Booted secondary processor 0x0000030301 [0x410fd082]
[ 2.793969] Detected PIPT I-cache on CPU30
[ 2.793979] GICv3: CPU30: found redistributor 30302 region
1:0x000000006d480000
[ 2.794040] CPU30: using LPI pending table @0x0000041fb83b0000
[ 2.794184] CPU30: Booted secondary processor 0x0000030302 [0x410fd082]
[ 2.876604] Detected PIPT I-cache on CPU31
[ 2.876615] GICv3: CPU31: found redistributor 30303 region
1:0x000000006d4c0000
[ 2.876677] CPU31: using LPI pending table @0x0000041fb83c0000
[ 2.876818] CPU31: Booted secondary processor 0x0000030303 [0x410fd082]
[ 2.958920] Detected PIPT I-cache on CPU32
[ 2.958961] GICv3: CPU32: found redistributor 50000 region
2:0x000004004d100000
[ 2.958994] CPU32: using LPI pending table @0x0000041fb83e0000
[ 2.959149] CPU32: Booted secondary processor 0x0000050000 [0x410fd082]
[ 3.041547] Detected PIPT I-cache on CPU33
[ 3.041572] GICv3: CPU33: found redistributor 50001 region
2:0x000004004d140000
[ 3.041597] CPU33: using LPI pending table @0x0000041fb83f0000
[ 3.041703] CPU33: Booted secondary processor 0x0000050001 [0x410fd082]
[ 3.124194] Detected PIPT I-cache on CPU34
[ 3.124218] GICv3: CPU34: found redistributor 50002 region
2:0x000004004d180000
[ 3.124244] CPU34: using LPI pending table @0x0000041fb7850000
[ 3.124351] CPU34: Booted secondary processor 0x0000050002 [0x410fd082]
[ 3.206841] Detected PIPT I-cache on CPU35
[ 3.206866] GICv3: CPU35: found redistributor 50003 region
2:0x000004004d1c0000
[ 3.206892] CPU35: using LPI pending table @0x0000041fb7860000
[ 3.206998] CPU35: Booted secondary processor 0x0000050003 [0x410fd082]
[ 3.289488] Detected PIPT I-cache on CPU36
[ 3.289514] GICv3: CPU36: found redistributor 50100 region
2:0x000004004d200000
[ 3.289540] CPU36: using LPI pending table @0x0000041fb7870000
[ 3.289649] CPU36: Booted secondary processor 0x0000050100 [0x410fd082]
[ 3.372137] Detected PIPT I-cache on CPU37
[ 3.372162] GICv3: CPU37: found redistributor 50101 region
2:0x000004004d240000
[ 3.372188] CPU37: using LPI pending table @0x0000041fb78d0000
[ 3.372294] CPU37: Booted secondary processor 0x0000050101 [0x410fd082]
[ 3.454782] Detected PIPT I-cache on CPU38
[ 3.454808] GICv3: CPU38: found redistributor 50102 region
2:0x000004004d280000
[ 3.454833] CPU38: using LPI pending table @0x0000041fb78e0000
[ 3.454940] CPU38: Booted secondary processor 0x0000050102 [0x410fd082]
[ 3.537432] Detected PIPT I-cache on CPU39
[ 3.537457] GICv3: CPU39: found redistributor 50103 region
2:0x000004004d2c0000
[ 3.537483] CPU39: using LPI pending table @0x0000041fb78f0000
[ 3.537589] CPU39: Booted secondary processor 0x0000050103 [0x410fd082]
[ 3.620080] Detected PIPT I-cache on CPU40
[ 3.620106] GICv3: CPU40: found redistributor 50200 region
2:0x000004004d300000
[ 3.620134] CPU40: using LPI pending table @0x0000041fb7940000
[ 3.620244] CPU40: Booted secondary processor 0x0000050200 [0x410fd082]
[ 3.702727] Detected PIPT I-cache on CPU41
[ 3.702752] GICv3: CPU41: found redistributor 50201 region
2:0x000004004d340000
[ 3.702778] CPU41: using LPI pending table @0x0000041fb7950000
[ 3.702885] CPU41: Booted secondary processor 0x0000050201 [0x410fd082]
[ 3.785375] Detected PIPT I-cache on CPU42
[ 3.785400] GICv3: CPU42: found redistributor 50202 region
2:0x000004004d380000
[ 3.785427] CPU42: using LPI pending table @0x0000041fb7960000
[ 3.785534] CPU42: Booted secondary processor 0x0000050202 [0x410fd082]
[ 3.868026] Detected PIPT I-cache on CPU43
[ 3.868052] GICv3: CPU43: found redistributor 50203 region
2:0x000004004d3c0000
[ 3.868080] CPU43: using LPI pending table @0x0000041fb79c0000
[ 3.868187] CPU43: Booted secondary processor 0x0000050203 [0x410fd082]
[ 3.950673] Detected PIPT I-cache on CPU44
[ 3.950700] GICv3: CPU44: found redistributor 50300 region
2:0x000004004d400000
[ 3.950728] CPU44: using LPI pending table @0x0000041fb79d0000
[ 3.950837] CPU44: Booted secondary processor 0x0000050300 [0x410fd082]
[ 4.033322] Detected PIPT I-cache on CPU45
[ 4.033348] GICv3: CPU45: found redistributor 50301 region
2:0x000004004d440000
[ 4.033374] CPU45: using LPI pending table @0x0000041fb79e0000
[ 4.033480] CPU45: Booted secondary processor 0x0000050301 [0x410fd082]
[ 4.115970] Detected PIPT I-cache on CPU46
[ 4.115996] GICv3: CPU46: found redistributor 50302 region
2:0x000004004d480000
[ 4.116023] CPU46: using LPI pending table @0x0000041fb7a40000
[ 4.116131] CPU46: Booted secondary processor 0x0000050302 [0x410fd082]
[ 4.198621] Detected PIPT I-cache on CPU47
[ 4.198647] GICv3: CPU47: found redistributor 50303 region
2:0x000004004d4c0000
[ 4.198674] CPU47: using LPI pending table @0x0000041fb7a50000
[ 4.198779] CPU47: Booted secondary processor 0x0000050303 [0x410fd082]
[ 4.280925] Detected PIPT I-cache on CPU48
[ 4.280958] GICv3: CPU48: found redistributor 70000 region
3:0x000004006d100000
[ 4.280996] CPU48: using LPI pending table @0x0000041fb7a60000
[ 4.281148] CPU48: Booted secondary processor 0x0000070000 [0x410fd082]
[ 4.363577] Detected PIPT I-cache on CPU49
[ 4.363606] GICv3: CPU49: found redistributor 70001 region
3:0x000004006d140000
[ 4.363643] CPU49: using LPI pending table @0x0000041fb7ae0000
[ 4.363788] CPU49: Booted secondary processor 0x0000070001 [0x410fd082]
[ 4.446225] Detected PIPT I-cache on CPU50
[ 4.446254] GICv3: CPU50: found redistributor 70002 region
3:0x000004006d180000
[ 4.446291] CPU50: using LPI pending table @0x0000041fb7af0000
[ 4.446438] CPU50: Booted secondary processor 0x0000070002 [0x410fd082]
[ 4.528875] Detected PIPT I-cache on CPU51
[ 4.528904] GICv3: CPU51: found redistributor 70003 region
3:0x000004006d1c0000
[ 4.528942] CPU51: using LPI pending table @0x0000041fb7b00000
[ 4.529089] CPU51: Booted secondary processor 0x0000070003 [0x410fd082]
[ 4.611527] Detected PIPT I-cache on CPU52
[ 4.611559] GICv3: CPU52: found redistributor 70100 region
3:0x000004006d200000
[ 4.611598] CPU52: using LPI pending table @0x0000041fb7b10000
[ 4.611750] CPU52: Booted secondary processor 0x0000070100 [0x410fd082]
[ 4.694176] Detected PIPT I-cache on CPU53
[ 4.694205] GICv3: CPU53: found redistributor 70101 region
3:0x000004006d240000
[ 4.694242] CPU53: using LPI pending table @0x0000041fb7b20000
[ 4.694389] CPU53: Booted secondary processor 0x0000070101 [0x410fd082]
[ 4.776826] Detected PIPT I-cache on CPU54
[ 4.776855] GICv3: CPU54: found redistributor 70102 region
3:0x000004006d280000
[ 4.776892] CPU54: using LPI pending table @0x0000041fb7b30000
[ 4.777039] CPU54: Booted secondary processor 0x0000070102 [0x410fd082]
[ 4.859476] Detected PIPT I-cache on CPU55
[ 4.859505] GICv3: CPU55: found redistributor 70103 region
3:0x000004006d2c0000
[ 4.859543] CPU55: using LPI pending table @0x0000041fb7b40000
[ 4.859689] CPU55: Booted secondary processor 0x0000070103 [0x410fd082]
[ 4.942127] Detected PIPT I-cache on CPU56
[ 4.942160] GICv3: CPU56: found redistributor 70200 region
3:0x000004006d300000
[ 4.942197] CPU56: using LPI pending table @0x0000041fb7b50000
[ 4.942352] CPU56: Booted secondary processor 0x0000070200 [0x410fd082]
[ 5.024775] Detected PIPT I-cache on CPU57
[ 5.024804] GICv3: CPU57: found redistributor 70201 region
3:0x000004006d340000
[ 5.024841] CPU57: using LPI pending table @0x0000041fb7b60000
[ 5.024988] CPU57: Booted secondary processor 0x0000070201 [0x410fd082]
[ 5.107423] Detected PIPT I-cache on CPU58
[ 5.107453] GICv3: CPU58: found redistributor 70202 region
3:0x000004006d380000
[ 5.107489] CPU58: using LPI pending table @0x0000041fb7b70000
[ 5.107635] CPU58: Booted secondary processor 0x0000070202 [0x410fd082]
[ 5.190075] Detected PIPT I-cache on CPU59
[ 5.190105] GICv3: CPU59: found redistributor 70203 region
3:0x000004006d3c0000
[ 5.190143] CPU59: using LPI pending table @0x0000041fb7b80000
[ 5.190289] CPU59: Booted secondary processor 0x0000070203 [0x410fd082]
[ 5.272725] Detected PIPT I-cache on CPU60
[ 5.272758] GICv3: CPU60: found redistributor 70300 region
3:0x000004006d400000
[ 5.272795] CPU60: using LPI pending table @0x0000041fb7b90000
[ 5.272950] CPU60: Booted secondary processor 0x0000070300 [0x410fd082]
[ 5.355374] Detected PIPT I-cache on CPU61
[ 5.355403] GICv3: CPU61: found redistributor 70301 region
3:0x000004006d440000
[ 5.355441] CPU61: using LPI pending table @0x0000041fb7ba0000
[ 5.355587] CPU61: Booted secondary processor 0x0000070301 [0x410fd082]
[ 5.438022] Detected PIPT I-cache on CPU62
[ 5.438052] GICv3: CPU62: found redistributor 70302 region
3:0x000004006d480000
[ 5.438089] CPU62: using LPI pending table @0x0000041fb7bb0000
[ 5.438235] CPU62: Booted secondary processor 0x0000070302 [0x410fd082]
[ 5.520675] Detected PIPT I-cache on CPU63
[ 5.520705] GICv3: CPU63: found redistributor 70303 region
3:0x000004006d4c0000
[ 5.520743] CPU63: using LPI pending table @0x0000041fb7bc0000
[ 5.520888] CPU63: Booted secondary processor 0x0000070303 [0x410fd082]
[ 5.520975] smp: Brought up 4 nodes, 64 CPUs
[ 7.042259] SMP: Total of 64 processors activated.
[ 7.047105] CPU features: detected: GIC system register CPU interface
[ 7.053625] CPU features: detected: 32-bit EL0 Support
[ 7.145591] CPU: All CPU(s) started at EL2
[ 7.149856] alternatives: patching kernel code
[ 7.154740] ERROR: Node-distance not symmetric
[ 7.154740]
[ 7.160724] 10 15 20 25
[ 7.163456] 15 10 25 30
[ 7.166190] 20 25 10 15
[ 7.168921] 10 10 15 10
[ 7.171655]
[ 7.175589] ERROR: groups don't span domain->span
[ 7.180352] ERROR: groups don't span domain->span
[ 7.185164] ERROR: groups don't span domain->span
[ 7.189926] ERROR: groups don't span domain->span
[ 7.194734] ERROR: groups don't span domain->span
[ 7.199495] ERROR: groups don't span domain->span
[ 7.204303] ERROR: groups don't span domain->span
[ 7.209064] ERROR: groups don't span domain->span
[ 7.213872] ERROR: groups don't span domain->span
[ 7.218632] ERROR: groups don't span domain->span
[ 7.223440] ERROR: groups don't span domain->span
[ 7.228201] ERROR: groups don't span domain->span
[ 7.233009] ERROR: groups don't span domain->span
[ 7.237770] ERROR: groups don't span domain->span
[ 7.242577] ERROR: groups don't span domain->span
[ 7.247337] ERROR: groups don't span domain->span
[ 7.252144] ERROR: groups don't span domain->span
[ 7.256905] ERROR: groups don't span domain->span
[ 7.261714] ERROR: groups don't span domain->span
[ 7.266474] ERROR: groups don't span domain->span
[ 7.271282] ERROR: groups don't span domain->span
[ 7.276042] ERROR: groups don't span domain->span
[ 7.280851] ERROR: groups don't span domain->span
[ 7.285611] ERROR: groups don't span domain->span
[ 7.290420] ERROR: groups don't span domain->span
[ 7.295180] ERROR: groups don't span domain->span
[ 7.299988] ERROR: groups don't span domain->span
[ 7.304748] ERROR: groups don't span domain->span
[ 7.309555] ERROR: groups don't span domain->span
[ 7.314315] ERROR: groups don't span domain->span
[ 7.319122] ERROR: groups don't span domain->span
[ 7.323883] ERROR: groups don't span domain->span
[ 7.328642] Unable to handle kernel NULL pointer dereference at
virtual address 0000000000000000
[ 7.337525] Mem abort info:
[ 7.340339] ESR = 0x96000004
[ 7.343421] Exception class = DABT (current EL), IL = 32 bits
[ 7.349403] SET = 0, FnV = 0
[ 7.352482] EA = 0, S1PTW = 0
[ 7.355651] Data abort info:
[ 7.358554] ISV = 0, ISS = 0x00000004
[ 7.362427] CM = 0, WnR = 0
[ 7.365418] [0000000000000000] user address but active_mm is swapper
[ 7.371840] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 7.377468] Modules linked in:
[ 7.380550] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W
4.19.0-rc6-00002-gea67197-dirty #877
[ 7.390308] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 7.396815] pstate: 20000005 (nzCv daif -PAN -UAO)
[ 7.401659] pc : __ll_sc_atomic_sub_return+0x4/0x20
[ 7.406585] lr : free_sched_groups.part.1+0x40/0x98
[ 7.411509] sp : ffff00000947bc40
[ 7.414851] x29: ffff00000947bc40 x28: ffff801fb4eae000
[ 7.420216] x27: 0000000000000010 x26: ffff000009119000
[ 7.425581] x25: 0000000000000010 x24: ffff801fb4e30000
[ 7.430946] x23: ffffffffffffe830 x22: 0000000000000001
[ 7.436311] x21: ffff801fb4e83000 x20: ffff801fb4eddf00
[ 7.441675] x19: ffff801fb4eddf80 x18: ffffffffffffffff
[ 7.447040] x17: 00000000ffffffff x16: 0000000000000000
[ 7.452405] x15: ffff0000091196c8 x14: ffff0000892746ef
[ 7.457770] x13: ffff0000092746fd x12: ffff000009119940
[ 7.463134] x11: ffff801fb62258b8 x10: 0000000000000001
[ 7.468499] x9 : ffff801fb49e4100 x8 : 000000000000000f
[ 7.473865] x7 : ffff801fb4d25700 x6 : 0000801ff2eea000
[ 7.479229] x5 : ffff0000091196f0 x4 : 0000000000000400
[ 7.484594] x3 : ffff801fb4edd000 x2 : ffff801fb4e83000
[ 7.489959] x1 : 0000000000000000 x0 : 0000000000000001
[ 7.495325] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
[ 7.502094] Call trace:
[ 7.504559] __ll_sc_atomic_sub_return+0x4/0x20
[ 7.509133] destroy_sched_domain+0x20/0x70
[ 7.513354] cpu_attach_domain+0xd8/0x688
[ 7.517401] build_sched_domains+0xd60/0xe38
[ 7.521709] sched_init_domains+0x6c/0x98
[ 7.525757] sched_init_smp+0x2c/0x7c
[ 7.529452] kernel_init_freeable+0xdc/0x244
[ 7.533762] kernel_init+0x10/0x108
[ 7.537281] ret_from_fork+0x10/0x18
[ 7.540888] Code: 88107c31 35ffffb0 d65f03c0 f9800031 (885f7c31)
[ 7.547113] ---[ end trace 137e0781b6e939ab ]---
[ 7.551789] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 7.551789]
[ 7.561033] SMP: stopping secondary CPUs
[ 7.565010] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b
[ 7.565010] ]---
I also note that if I apply the patch, below, to reject the invalid NUMA
distance, we're still getting a warning/error:
[ 7.144407] CPU: All CPU(s) started at EL2
[ 7.148678] alternatives: patching kernel code
[ 7.153557] ERROR: Node-0 not representative
[ 7.153557]
[ 7.159365] 10 15 20 25
[ 7.162097] 15 10 25 30
[ 7.164832] 20 25 10 15
[ 7.167562] 25 30 15 10
[ 7.170296]
[ 7.175103] ERROR: groups don't span domain->span
[ 7.179929] ERROR: groups don't span domain->span
[ 7.184751] ERROR: groups don't span domain->span
[ 7.189572] ERROR: groups don't span domain->span
[ 7.194392] ERROR: groups don't span domain->span
[ 7.199212] ERROR: groups don't span domain->span
[ 7.204032] ERROR: groups don't span domain->span
[ 7.208851] ERROR: groups don't span domain->span
[ 7.213672] ERROR: groups don't span domain->span
[ 7.218492] ERROR: groups don't span domain->span
[ 7.223313] ERROR: groups don't span domain->span
[ 7.228133] ERROR: groups don't span domain->span
[ 7.232952] ERROR: groups don't span domain->span
[ 7.237773] ERROR: groups don't span domain->span
[ 7.242591] ERROR: groups don't span domain->span
[ 7.247412] ERROR: groups don't span domain->span
[ 7.252238] ERROR: groups don't span domain->span
[ 7.257063] ERROR: groups don't span domain->span
[ 7.261888] ERROR: groups don't span domain->span
[ 7.266712] ERROR: groups don't span domain->span
[ 7.271536] ERROR: groups don't span domain->span
[ 7.276361] ERROR: groups don't span domain->span
[ 7.281185] ERROR: groups don't span domain->span
[ 7.286010] ERROR: groups don't span domain->span
[ 7.290835] ERROR: groups don't span domain->span
[ 7.295659] ERROR: groups don't span domain->span
[ 7.300483] ERROR: groups don't span domain->span
[ 7.305307] ERROR: groups don't span domain->span
[ 7.310132] ERROR: groups don't span domain->span
[ 7.314957] ERROR: groups don't span domain->span
[ 7.319782] ERROR: groups don't span domain->span
[ 7.324609] ERROR: groups don't span domain->span
[ 7.329432] ERROR: groups don't span domain->span
[ 7.334254] ERROR: groups don't span domain->span
[ 7.339080] ERROR: groups don't span domain->span
[ 7.343903] ERROR: groups don't span domain->span
[ 7.348725] ERROR: groups don't span domain->span
[ 7.353547] ERROR: groups don't span domain->span
[ 7.358368] ERROR: groups don't span domain->span
[ 7.363191] ERROR: groups don't span domain->span
[ 7.368012] ERROR: groups don't span domain->span
[ 7.372835] ERROR: groups don't span domain->span
[ 7.377658] ERROR: groups don't span domain->span
[ 7.382480] ERROR: groups don't span domain->span
[ 7.387303] ERROR: groups don't span domain->span
[ 7.392124] ERROR: groups don't span domain->span
[ 7.396949] ERROR: groups don't span domain->span
[ 7.401771] ERROR: groups don't span domain->span
[ 7.406601] ERROR: groups don't span domain->span
[ 7.411428] ERROR: groups don't span domain->span
[ 7.416256] ERROR: groups don't span domain->span
[ 7.421086] ERROR: groups don't span domain->span
[ 7.425913] ERROR: groups don't span domain->span
[ 7.430740] ERROR: groups don't span domain->span
[ 7.435566] ERROR: groups don't span domain->span
[ 7.440393] ERROR: groups don't span domain->span
[ 7.445220] ERROR: groups don't span domain->span
[ 7.450047] ERROR: groups don't span domain->span
[ 7.454877] ERROR: groups don't span domain->span
[ 7.459705] ERROR: groups don't span domain->span
[ 7.464533] ERROR: groups don't span domain->span
[ 7.469361] ERROR: groups don't span domain->span
[ 7.474189] ERROR: groups don't span domain->span
[ 7.479016] ERROR: groups don't span domain->span
[ 7.483786] root domain span: 0-63 (max cpu_capacity = 1024)
[ 7.504868] devtmpfs: initialized
[ 7.510342] clocksource: jiffies: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 7645041785100000 ns
John
>
>>> However I am interested to know if the scheduler crash is a real
>>> problem.
>>
>> Ideally we'd not crash of course.. let me see if I can still make sense
>> of that topology code.
>
> JFYI, https://lkml.org/lkml/2018/10/26/527
>
> So it seems to be that setting the distance between separate nodes to
> LOCAL_DISTANCE causes the problem. x86 NUMA code - like arm64 - allows
> this. However ACPI SLIT validation rejects this, unlike OF distance map
> parsing.
>
> Thanks,
> John
>
> _______________________________________________
> Linuxarm mailing list
> Linuxarm@huawei.com
> http://hulk.huawei.com/mailman/listinfo/linuxarm
>
> .
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-10-30 15:35 ` John Garry
@ 2018-10-31 20:46 ` Peter Zijlstra
2018-11-01 10:01 ` John Garry
0 siblings, 1 reply; 13+ messages in thread
From: Peter Zijlstra @ 2018-10-31 20:46 UTC (permalink / raw)
To: John Garry
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
linux-kernel, Linuxarm, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On Tue, Oct 30, 2018 at 03:35:35PM +0000, John Garry wrote:
> [ 7.154740] ERROR: Node-distance not symmetric
> [ 7.154740]
> [ 7.160724] 10 15 20 25
> [ 7.163456] 15 10 25 30
> [ 7.166190] 20 25 10 15
> [ 7.168921] 10 10 15 10
> [ 7.171655]
But I'm not getting the rest of those errors with my 'reproducer':
kvm -smp 4 -m 4G -display none -monitor null -serial stdio -kernel defconfig-build/arch/x86/boot/bzImage -append "sched_debug debug ignore_loglevel earlyprintk=serial,ttyS0,115200,keep numa=fake=4:10,15,20,25,15,10,25,30,20,25,10,15,10,10,15,10,0"
[ 0.828331] ERROR: Node-distance not symmetric
[ 0.828331]
[ 0.829081] 10 15 20 25
[ 0.830079] 15 10 25 30
[ 0.831079] 20 25 10 15
[ 0.832079] 10 10 15 10
[ 0.833079]
[ 0.834373] CPU0 attaching sched-domain(s):
[ 0.835082] domain-0: span=0-3 level=DIE
[ 0.836079] groups: 0:{ span=0 }, 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }
[ 0.837082] CPU1 attaching sched-domain(s):
[ 0.838081] domain-0: span=0-3 level=DIE
[ 0.839079] groups: 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }
[ 0.840082] CPU2 attaching sched-domain(s):
[ 0.841080] domain-0: span=0-3 level=DIE
[ 0.842079] groups: 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }, 1:{ span=1 }
[ 0.843094] ------------[ cut here ]------------
[ 0.844076] kernel BUG at ../mm/slub.c:3901!
[ 0.844083] invalid opcode: 0000 [#1] SMP PTI
[ 0.845076] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.0-rc8+ #305
[ 0.845076] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
[ 0.845076] RIP: 0010:kfree+0x113/0x160
[ 0.845076] Code: 18 48 89 da 4c 89 e6 e8 db 01 c5 00 48 8b 45 00 48 85 c0 75 e4 e9 0e ff ff ff 49 8b 02 f6 c4 80 75 0a 49 8b 42 08 a8 01 75 02 <0f> 0b 49 8b 02 31 f6 f6 c4 80 74 05 41 0f b6 72 51 5b 5d 41 5c 4c
[ 0.845076] RSP: 0000:ffffabc080633dc8 EFLAGS: 00010246
[ 0.845076] RAX: ffff9f973fff8da0 RBX: ffff9f970000001e RCX: 00000000000000f9
[ 0.845076] RDX: 0000000000000000 RSI: ffff9f963ea23c80 RDI: 0000606980000000
[ 0.845076] RBP: 0000000000020ac0 R08: 0000000000023c80 R09: ffffffff9f8a10db
[ 0.845076] R10: fffff17204000000 R11: 0000000000000001 R12: ffffffff9f8a113d
[ 0.845076] R13: 0000000000000003 R14: ffffffffa0ab4820 R15: ffff9f973e5bde00
[ 0.845076] FS: 0000000000000000(0000) GS:ffff9f963ea00000(0000) knlGS:0000000000000000
[ 0.845076] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 0.845076] CR2: 00000000ffffffff CR3: 000000008ea0a000 CR4: 00000000000006f0
[ 0.845076] Call Trace:
[ 0.845076] destroy_sched_domain+0x3d/0x50
[ 0.845076] cpu_attach_domain+0x378/0x680
[ 0.845076] ? update_group_capacity+0x20/0x2c0
[ 0.845076] build_sched_domains+0xde9/0xed0
[ 0.845076] ? set_debug_rodata+0xc/0xc
[ 0.845076] sched_init_domains+0x80/0x90
[ 0.845076] sched_init_smp+0x1d/0x63
[ 0.845076] kernel_init_freeable+0x101/0x23f
[ 0.845076] ? rest_init+0xb0/0xb0
[ 0.845076] kernel_init+0x5/0x100
[ 0.845076] ret_from_fork+0x35/0x40
I'll work on that crash though..
> I also note that if I apply the patch, below, to reject the invalid NUMA
> distance, we're still getting a warning/error:
>
> [ 7.144407] CPU: All CPU(s) started at EL2
> [ 7.148678] alternatives: patching kernel code
> [ 7.153557] ERROR: Node-0 not representative
> [ 7.153557]
> [ 7.159365] 10 15 20 25
> [ 7.162097] 15 10 25 30
> [ 7.164832] 20 25 10 15
> [ 7.167562] 25 30 15 10
Yeah, that's an 'obviously' broken topology too.
Clearly you're far more creative than the ACPI BIOS people have been so
far.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-10-31 20:46 ` Peter Zijlstra
@ 2018-11-01 10:01 ` John Garry
2018-11-02 9:39 ` Peter Zijlstra
2018-11-02 10:50 ` Peter Zijlstra
0 siblings, 2 replies; 13+ messages in thread
From: John Garry @ 2018-11-01 10:01 UTC (permalink / raw)
To: Peter Zijlstra
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
linux-kernel, Linuxarm, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On 31/10/2018 20:46, Peter Zijlstra wrote:
> On Tue, Oct 30, 2018 at 03:35:35PM +0000, John Garry wrote:
>> [ 7.154740] ERROR: Node-distance not symmetric
>> [ 7.154740]
>> [ 7.160724] 10 15 20 25
>> [ 7.163456] 15 10 25 30
>> [ 7.166190] 20 25 10 15
>> [ 7.168921] 10 10 15 10
>> [ 7.171655]
>
> But I'm not getting the rest of those errors with my 'reproducer':
>
> kvm -smp 4 -m 4G -display none -monitor null -serial stdio -kernel defconfig-build/arch/x86/boot/bzImage -append "sched_debug debug ignore_loglevel earlyprintk=serial,ttyS0,115200,keep numa=fake=4:10,15,20,25,15,10,25,30,20,25,10,15,10,10,15,10,0"
>
> [ 0.828331] ERROR: Node-distance not symmetric
> [ 0.828331]
> [ 0.829081] 10 15 20 25
> [ 0.830079] 15 10 25 30
> [ 0.831079] 20 25 10 15
> [ 0.832079] 10 10 15 10
> [ 0.833079]
> [ 0.834373] CPU0 attaching sched-domain(s):
> [ 0.835082] domain-0: span=0-3 level=DIE
> [ 0.836079] groups: 0:{ span=0 }, 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }
> [ 0.837082] CPU1 attaching sched-domain(s):
> [ 0.838081] domain-0: span=0-3 level=DIE
> [ 0.839079] groups: 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }
> [ 0.840082] CPU2 attaching sched-domain(s):
> [ 0.841080] domain-0: span=0-3 level=DIE
> [ 0.842079] groups: 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }, 1:{ span=1 }
> [ 0.843094] ------------[ cut here ]------------
> [ 0.844076] kernel BUG at ../mm/slub.c:3901!
> [ 0.844083] invalid opcode: 0000 [#1] SMP PTI
> [ 0.845076] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.0-rc8+ #305
> [ 0.845076] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
> [ 0.845076] RIP: 0010:kfree+0x113/0x160
> [ 0.845076] Code: 18 48 89 da 4c 89 e6 e8 db 01 c5 00 48 8b 45 00 48 85 c0 75 e4 e9 0e ff ff ff 49 8b 02 f6 c4 80 75 0a 49 8b 42 08 a8 01 75 02 <0f> 0b 49 8b 02 31 f6 f6 c4 80 74 05 41 0f b6 72 51 5b 5d 41 5c 4c
> [ 0.845076] RSP: 0000:ffffabc080633dc8 EFLAGS: 00010246
> [ 0.845076] RAX: ffff9f973fff8da0 RBX: ffff9f970000001e RCX: 00000000000000f9
> [ 0.845076] RDX: 0000000000000000 RSI: ffff9f963ea23c80 RDI: 0000606980000000
> [ 0.845076] RBP: 0000000000020ac0 R08: 0000000000023c80 R09: ffffffff9f8a10db
> [ 0.845076] R10: fffff17204000000 R11: 0000000000000001 R12: ffffffff9f8a113d
> [ 0.845076] R13: 0000000000000003 R14: ffffffffa0ab4820 R15: ffff9f973e5bde00
> [ 0.845076] FS: 0000000000000000(0000) GS:ffff9f963ea00000(0000) knlGS:0000000000000000
> [ 0.845076] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 0.845076] CR2: 00000000ffffffff CR3: 000000008ea0a000 CR4: 00000000000006f0
> [ 0.845076] Call Trace:
> [ 0.845076] destroy_sched_domain+0x3d/0x50
> [ 0.845076] cpu_attach_domain+0x378/0x680
> [ 0.845076] ? update_group_capacity+0x20/0x2c0
> [ 0.845076] build_sched_domains+0xde9/0xed0
> [ 0.845076] ? set_debug_rodata+0xc/0xc
> [ 0.845076] sched_init_domains+0x80/0x90
> [ 0.845076] sched_init_smp+0x1d/0x63
> [ 0.845076] kernel_init_freeable+0x101/0x23f
> [ 0.845076] ? rest_init+0xb0/0xb0
> [ 0.845076] kernel_init+0x5/0x100
> [ 0.845076] ret_from_fork+0x35/0x40
>
> I'll work on that crash though..
The actual crash callchain seems to be
destroy_sched_domain()->free_sched_groups():
static void free_sched_groups(struct sched_group *sg, int free_sgc)
{
...
do {
tmp = sg->next;
if (free_sgc && atomic_dec_and_test(&sg->sgc->ref))***
kfree(sg->sgc);
...
}
*** crash occurs when free_sgc is non-zero and sg->sgc is NULL
And, as I mentioned earlier, I bisected this problem to 58d5af59d55b.
>
>> I also note that if I apply the patch, below, to reject the invalid NUMA
>> distance, we're still getting a warning/error:
>>
>> [ 7.144407] CPU: All CPU(s) started at EL2
>> [ 7.148678] alternatives: patching kernel code
>> [ 7.153557] ERROR: Node-0 not representative
>> [ 7.153557]
>> [ 7.159365] 10 15 20 25
>> [ 7.162097] 15 10 25 30
>> [ 7.164832] 20 25 10 15
>> [ 7.167562] 25 30 15 10
>
> Yeah, that's an 'obviously' broken topology too.
>
AFAICT, this conforms to ACPI spec SLIT rules, and the kernel SLIT
validation allows this also. So maybe we should shout louder here or
even mark the SLIT as invalid if totally broken.
> Clearly you're far more creative than the ACPI BIOS people have been so
> far.
Cheers,
John
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-11-01 10:01 ` John Garry
@ 2018-11-02 9:39 ` Peter Zijlstra
2018-11-02 10:10 ` John Garry
2018-11-02 10:50 ` Peter Zijlstra
1 sibling, 1 reply; 13+ messages in thread
From: Peter Zijlstra @ 2018-11-02 9:39 UTC (permalink / raw)
To: John Garry
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
linux-kernel, Linuxarm, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel, suravee.suthikulpanit
On Thu, Nov 01, 2018 at 10:01:05AM +0000, John Garry wrote:
> On 31/10/2018 20:46, Peter Zijlstra wrote:
> > On Tue, Oct 30, 2018 at 03:35:35PM +0000, John Garry wrote:
> > > [ 7.154740] ERROR: Node-distance not symmetric
> > > [ 7.154740]
> > > [ 7.160724] 10 15 20 25
> > > [ 7.163456] 15 10 25 30
> > > [ 7.166190] 20 25 10 15
> > > [ 7.168921] 10 10 15 10
> > > [ 7.171655]
> >
> > But I'm not getting the rest of those errors with my 'reproducer':
> >
> > kvm -smp 4 -m 4G -display none -monitor null -serial stdio -kernel defconfig-build/arch/x86/boot/bzImage -append "sched_debug debug ignore_loglevel earlyprintk=serial,ttyS0,115200,keep numa=fake=4:10,15,20,25,15,10,25,30,20,25,10,15,10,10,15,10,0"
> >
> > [ 0.828331] ERROR: Node-distance not symmetric
> > [ 0.828331]
> > [ 0.829081] 10 15 20 25
> > [ 0.830079] 15 10 25 30
> > [ 0.831079] 20 25 10 15
> > [ 0.832079] 10 10 15 10
> > [ 0.833079]
> > [ 0.834373] CPU0 attaching sched-domain(s):
> > [ 0.835082] domain-0: span=0-3 level=DIE
> > [ 0.836079] groups: 0:{ span=0 }, 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }
> > [ 0.837082] CPU1 attaching sched-domain(s):
> > [ 0.838081] domain-0: span=0-3 level=DIE
> > [ 0.839079] groups: 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }
> > [ 0.840082] CPU2 attaching sched-domain(s):
> > [ 0.841080] domain-0: span=0-3 level=DIE
> > [ 0.842079] groups: 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }, 1:{ span=1 }
> > [ 0.843094] ------------[ cut here ]------------
> > [ 0.844076] kernel BUG at ../mm/slub.c:3901!
> > [ 0.844083] invalid opcode: 0000 [#1] SMP PTI
> > [ 0.845076] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.19.0-rc8+ #305
> > [ 0.845076] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.10.2-1 04/01/2014
> > [ 0.845076] RIP: 0010:kfree+0x113/0x160
> > [ 0.845076] Code: 18 48 89 da 4c 89 e6 e8 db 01 c5 00 48 8b 45 00 48 85 c0 75 e4 e9 0e ff ff ff 49 8b 02 f6 c4 80 75 0a 49 8b 42 08 a8 01 75 02 <0f> 0b 49 8b 02 31 f6 f6 c4 80 74 05 41 0f b6 72 51 5b 5d 41 5c 4c
> > [ 0.845076] RSP: 0000:ffffabc080633dc8 EFLAGS: 00010246
> > [ 0.845076] RAX: ffff9f973fff8da0 RBX: ffff9f970000001e RCX: 00000000000000f9
> > [ 0.845076] RDX: 0000000000000000 RSI: ffff9f963ea23c80 RDI: 0000606980000000
> > [ 0.845076] RBP: 0000000000020ac0 R08: 0000000000023c80 R09: ffffffff9f8a10db
> > [ 0.845076] R10: fffff17204000000 R11: 0000000000000001 R12: ffffffff9f8a113d
> > [ 0.845076] R13: 0000000000000003 R14: ffffffffa0ab4820 R15: ffff9f973e5bde00
> > [ 0.845076] FS: 0000000000000000(0000) GS:ffff9f963ea00000(0000) knlGS:0000000000000000
> > [ 0.845076] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > [ 0.845076] CR2: 00000000ffffffff CR3: 000000008ea0a000 CR4: 00000000000006f0
> > [ 0.845076] Call Trace:
> > [ 0.845076] destroy_sched_domain+0x3d/0x50
> > [ 0.845076] cpu_attach_domain+0x378/0x680
> > [ 0.845076] ? update_group_capacity+0x20/0x2c0
> > [ 0.845076] build_sched_domains+0xde9/0xed0
> > [ 0.845076] ? set_debug_rodata+0xc/0xc
> > [ 0.845076] sched_init_domains+0x80/0x90
> > [ 0.845076] sched_init_smp+0x1d/0x63
> > [ 0.845076] kernel_init_freeable+0x101/0x23f
> > [ 0.845076] ? rest_init+0xb0/0xb0
> > [ 0.845076] kernel_init+0x5/0x100
> > [ 0.845076] ret_from_fork+0x35/0x40
> >
> > I'll work on that crash though..
>
> The actual crash callchain seems to be
> destroy_sched_domain()->free_sched_groups():
>
> static void free_sched_groups(struct sched_group *sg, int free_sgc)
> {
> ...
> do {
> tmp = sg->next;
>
> if (free_sgc && atomic_dec_and_test(&sg->sgc->ref))***
> kfree(sg->sgc);
>
> ...
> }
>
> *** crash occurs when free_sgc is non-zero and sg->sgc is NULL
Yeah, turns out to be random memory corruption; I've had the crash in a
number of weird places; also GCC version dependent.
KASAN is awesome and pinpointed the problem though.
> And, as I mentioned earlier, I bisected this problem to 58d5af59d55b.
You mean:
051f3ca02e46 ("sched/topology: Introduce NUMA identity node sched domain")
right? and yes indeed! The below fixes my reproducer:
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 9d74371e4aad..039578429c25 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1337,7 +1348,7 @@ void sched_init_numa(void)
int level = 0;
int i, j, k;
- sched_domains_numa_distance = kzalloc(sizeof(int) * nr_node_ids, GFP_KERNEL);
+ sched_domains_numa_distance = kzalloc(sizeof(int) * (nr_node_ids + 1), GFP_KERNEL);
if (!sched_domains_numa_distance)
return;
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-11-02 9:39 ` Peter Zijlstra
@ 2018-11-02 10:10 ` John Garry
0 siblings, 0 replies; 13+ messages in thread
From: John Garry @ 2018-11-02 10:10 UTC (permalink / raw)
To: Peter Zijlstra
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
linux-kernel, Linuxarm, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel, suravee.suthikulpanit
>>
>> static void free_sched_groups(struct sched_group *sg, int free_sgc)
>> {
>> ...
>> do {
>> tmp = sg->next;
>>
>> if (free_sgc && atomic_dec_and_test(&sg->sgc->ref))***
>> kfree(sg->sgc);
>>
>> ...
>> }
>>
>> *** crash occurs when free_sgc is non-zero and sg->sgc is NULL
>
> Yeah, turns out to be random memory corruption; I've had the crash in a
> number of weird places; also GCC version dependent.
>
> KASAN is awesome and pinpointed the problem though.
>
>> And, as I mentioned earlier, I bisected this problem to 58d5af59d55b.
>
> You mean:
>
> 051f3ca02e46 ("sched/topology: Introduce NUMA identity node sched domain")
>
> right? and yes indeed! The below fixes my reproducer:
>
Yes, that's the one.
>
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 9d74371e4aad..039578429c25 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -1337,7 +1348,7 @@ void sched_init_numa(void)
> int level = 0;
> int i, j, k;
>
> - sched_domains_numa_distance = kzalloc(sizeof(int) * nr_node_ids, GFP_KERNEL);
> + sched_domains_numa_distance = kzalloc(sizeof(int) * (nr_node_ids + 1), GFP_KERNEL);
Good find.
> if (!sched_domains_numa_distance)
> return;
>
>
So what about this:
>>> I also note that if I apply the patch, below, to reject the invalid
NUMA
>>> distance, we're still getting a warning/error:
>>>
>>> [ 7.144407] CPU: All CPU(s) started at EL2
>>> [ 7.148678] alternatives: patching kernel code
>>> [ 7.153557] ERROR: Node-0 not representative
>>> [ 7.153557]
>>> [ 7.159365] 10 15 20 25
>>> [ 7.162097] 15 10 25 30
>>> [ 7.164832] 20 25 10 15
>>> [ 7.167562] 25 30 15 10
>>
>> Yeah, that's an 'obviously' broken topology too.
>>
>
> AFAICT, this conforms to ACPI spec SLIT rules, and the kernel SLIT
> validation allows this also. So maybe we should shout louder here or
> even mark the SLIT as invalid if totally broken.
>
I plan to fix up OF map parsing to reject invalid distance maps.
However is this distance map so broken for the scheduler that it's
better to reject this "valid" distance map also? If not, it may be nice
for the user to know about it without having to enable scheduler debugging.
Thanks,
John
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-11-01 10:01 ` John Garry
2018-11-02 9:39 ` Peter Zijlstra
@ 2018-11-02 10:50 ` Peter Zijlstra
2018-11-02 12:08 ` John Garry
1 sibling, 1 reply; 13+ messages in thread
From: Peter Zijlstra @ 2018-11-02 10:50 UTC (permalink / raw)
To: John Garry
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
linux-kernel, Linuxarm, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On Thu, Nov 01, 2018 at 10:01:05AM +0000, John Garry wrote:
> On 31/10/2018 20:46, Peter Zijlstra wrote:
> > > I also note that if I apply the patch, below, to reject the invalid NUMA
> > > distance, we're still getting a warning/error:
> > >
> > > [ 7.144407] CPU: All CPU(s) started at EL2
> > > [ 7.148678] alternatives: patching kernel code
> > > [ 7.153557] ERROR: Node-0 not representative
> > > [ 7.153557]
> > > [ 7.159365] 10 15 20 25
> > > [ 7.162097] 15 10 25 30
> > > [ 7.164832] 20 25 10 15
> > > [ 7.167562] 25 30 15 10
> >
> > Yeah, that's an 'obviously' broken topology too.
> >
>
> AFAICT, this conforms to ACPI spec SLIT rules, and the kernel SLIT
> validation allows this also. So maybe we should shout louder here or even
> mark the SLIT as invalid if totally broken.
Right. Part of the problem is that I only have a few necessary
conditions (symmetry, trace-identity, node0-complete-distance) for the
distance table, but together they are not sufficient to determine if a
topology is 'good'.
(also, strictly speaking, non-symmetric topologies are possible -- think
uni-directional mesh links -- we've just not ever seen and validated
those)
That said; I can certainly make this code give up and warn harder on
those conditions.
How is something like the below?
---
diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
index 9d74371e4aad..41e703dd875f 100644
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1207,6 +1218,11 @@ sd_init(struct sched_domain_topology_level *tl,
return sd;
}
+static const struct cpumask *cpu_system_mask(int cpu)
+{
+ return cpu_online_mask;
+}
+
/*
* Topology list, bottom-up.
*/
@@ -1337,7 +1353,7 @@ void sched_init_numa(void)
int level = 0;
int i, j, k;
- sched_domains_numa_distance = kzalloc(sizeof(int) * nr_node_ids, GFP_KERNEL);
+ sched_domains_numa_distance = kzalloc(sizeof(int) * (nr_node_ids + 1), GFP_KERNEL);
if (!sched_domains_numa_distance)
return;
@@ -1353,39 +1369,22 @@ void sched_init_numa(void)
* node_distance(i,j) in order to avoid cubic time.
*/
next_distance = curr_distance;
+
for (i = 0; i < nr_node_ids; i++) {
for (j = 0; j < nr_node_ids; j++) {
- for (k = 0; k < nr_node_ids; k++) {
- int distance = node_distance(i, k);
-
- if (distance > curr_distance &&
- (distance < next_distance ||
- next_distance == curr_distance))
- next_distance = distance;
-
- /*
- * While not a strong assumption it would be nice to know
- * about cases where if node A is connected to B, B is not
- * equally connected to A.
- */
- if (sched_debug() && node_distance(k, i) != distance)
- sched_numa_warn("Node-distance not symmetric");
-
- if (sched_debug() && i && !find_numa_distance(distance))
- sched_numa_warn("Node-0 not representative");
- }
- if (next_distance != curr_distance) {
- sched_domains_numa_distance[level++] = next_distance;
- sched_domains_numa_levels = level;
- curr_distance = next_distance;
- } else break;
- }
+ int distance = node_distance(0, j);
- /*
- * In case of sched_debug() we verify the above assumption.
- */
- if (!sched_debug())
- break;
+ if (distance > curr_distance &&
+ (distance < next_distance ||
+ next_distance == curr_distance))
+ next_distance = distance;
+
+ }
+ if (next_distance != curr_distance) {
+ sched_domains_numa_distance[level++] = next_distance;
+ sched_domains_numa_levels = level;
+ curr_distance = next_distance;
+ } else break;
}
/*
@@ -1428,7 +1427,24 @@ void sched_init_numa(void)
sched_domains_numa_masks[i][j] = mask;
for_each_node(k) {
- if (node_distance(j, k) > sched_domains_numa_distance[i])
+ int distance = node_distance(j, k);
+
+ if (!i && j != k) {
+ sched_numa_warn("Non-trace locality\n");
+ goto fail;
+ }
+
+ if (distance > curr_distance) {
+ sched_numa_warn("Node-0 not complete\n");
+ goto fail;
+ }
+
+ if (distance != node_distance(k, j)) {
+ sched_numa_warn("Non symmetric\n");
+ goto fail;
+ }
+
+ if (distance > sched_domains_numa_distance[i])
continue;
cpumask_or(mask, mask, cpumask_of_node(k));
@@ -1437,9 +1453,10 @@ void sched_init_numa(void)
}
/* Compute default topology size */
- for (i = 0; sched_domain_topology[i].mask; i++);
+ for (k = 0; sched_domain_topology[k].mask; k++)
+ ;
- tl = kzalloc((i + level + 1) *
+ tl = kzalloc((k + level + 1) *
sizeof(struct sched_domain_topology_level), GFP_KERNEL);
if (!tl)
return;
@@ -1447,7 +1464,7 @@ void sched_init_numa(void)
/*
* Copy the default topology bits..
*/
- for (i = 0; sched_domain_topology[i].mask; i++)
+ for (i = 0; i < k; i++)
tl[i] = sched_domain_topology[i];
/*
@@ -1478,6 +1495,31 @@ void sched_init_numa(void)
sched_max_numa_distance = sched_domains_numa_distance[level - 1];
init_numa_topology_type();
+
+ return;
+
+fail:
+ /* Compute default topology size */
+ for (k = 0; sched_domain_topology[k].mask; k++)
+ ;
+
+ tl = kzalloc((k + 2) *
+ sizeof(struct sched_domain_topology_level), GFP_KERNEL);
+ if (!tl)
+ return;
+
+ /*
+ * Copy the default topology bits..
+ */
+ for (i = 0; i < k; i++)
+ tl[i] = sched_domain_topology[i];
+
+ tl[i] = (struct sched_domain_topology_level){
+ .mask = cpu_system_mask,
+ SD_INIT_NAME(SYSTEM)
+ };
+
+ sched_domain_topology = tl;
}
void sched_domains_numa_masks_set(unsigned int cpu)
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-11-02 10:50 ` Peter Zijlstra
@ 2018-11-02 12:08 ` John Garry
2018-11-02 12:19 ` Peter Zijlstra
0 siblings, 1 reply; 13+ messages in thread
From: John Garry @ 2018-11-02 12:08 UTC (permalink / raw)
To: Peter Zijlstra
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
Linuxarm, linux-kernel, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On 02/11/2018 10:50, Peter Zijlstra wrote:
> On Thu, Nov 01, 2018 at 10:01:05AM +0000, John Garry wrote:
>> On 31/10/2018 20:46, Peter Zijlstra wrote:
>
>>>> I also note that if I apply the patch, below, to reject the invalid NUMA
>>>> distance, we're still getting a warning/error:
>>>>
>>>> [ 7.144407] CPU: All CPU(s) started at EL2
>>>> [ 7.148678] alternatives: patching kernel code
>>>> [ 7.153557] ERROR: Node-0 not representative
>>>> [ 7.153557]
>>>> [ 7.159365] 10 15 20 25
>>>> [ 7.162097] 15 10 25 30
>>>> [ 7.164832] 20 25 10 15
>>>> [ 7.167562] 25 30 15 10
>>>
>>> Yeah, that's an 'obviously' broken topology too.
>>>
>>
>> AFAICT, this conforms to ACPI spec SLIT rules, and the kernel SLIT
>> validation allows this also. So maybe we should shout louder here or even
>> mark the SLIT as invalid if totally broken.
>
> Right. Part of the problem is that I only have a few necessary
> conditions (symmetry, trace-identity, node0-complete-distance) for the
> distance table, but together they are not sufficient to determine if a
> topology is 'good'.
>
> (also, strictly speaking, non-symmetric topologies are possible -- think
> uni-directional mesh links -- we've just not ever seen and validated
> those)
>
> That said; I can certainly make this code give up and warn harder on
> those conditions.
That sounds reasonable, but I think that you would know better on the
policy of whether it's better to give up when conditions are not met.
Regardless, a louder warn sounds good.
>
> How is something like the below?
>
> ---
>
> diff --git a/kernel/sched/topology.c b/kernel/sched/topology.c
> index 9d74371e4aad..41e703dd875f 100644
> --- a/kernel/sched/topology.c
> +++ b/kernel/sched/topology.c
> @@ -1207,6 +1218,11 @@ sd_init(struct sched_domain_topology_level *tl,
> return sd;
> }
>
> +static const struct cpumask *cpu_system_mask(int cpu)
> +{
> + return cpu_online_mask;
> +}
> +
> /*
> * Topology list, bottom-up.
> */
> @@ -1337,7 +1353,7 @@ void sched_init_numa(void)
> int level = 0;
> int i, j, k;
>
> - sched_domains_numa_distance = kzalloc(sizeof(int) * nr_node_ids, GFP_KERNEL);
> + sched_domains_numa_distance = kzalloc(sizeof(int) * (nr_node_ids + 1), GFP_KERNEL);
Ahhhh, unfortuately this change in isolation is not fixing the crash for
me. I'm still crashing in the same place repeatedly.
Here's relevant log with KASAN enabled:
[ 10.655465]
==================================================================
[ 10.662772] BUG: KASAN: use-after-free in
free_sched_groups.part.1+0x4c/0xf0
[ 10.669896] Read of size 8 at addr ffff801db4668000 by task swapper/0/1
[ 10.676578]
[ 10.678079] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
4.19.0-rc6-00018-gf417e94-dirty #945
[ 10.686432] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 10.692940] Call trace:
[ 10.695407] dump_backtrace+0x0/0x230
[ 10.699104] show_stack+0x14/0x20
[ 10.702450] dump_stack+0xa0/0xc4
[ 10.705796] print_address_description+0x60/0x270
[ 10.710546] kasan_report+0x258/0x360
[ 10.714243] __asan_load8+0x84/0xa8
[ 10.717764] free_sched_groups.part.1+0x4c/0xf0
[ 10.722339] destroy_sched_domain+0x50/0x118
[ 10.726651] cpu_attach_domain+0x140/0x450
[ 10.730787] build_sched_domains+0x1750/0x1820
[ 10.735274] sched_init_domains+0x8c/0xb0
[ 10.739324] sched_init_smp+0x2c/0x7c
[ 10.743021] kernel_init_freeable+0x138/0x2f0
[ 10.747421] kernel_init+0x10/0x120
[ 10.750941] ret_from_fork+0x10/0x18
[ 10.754547]
[ 10.756046] Allocated by task 1:
[ 10.759303] kasan_kmalloc+0xd0/0x180
[ 10.763000] build_sched_domains+0x2d0/0x1820
[ 10.767400] sched_init_domains+0x8c/0xb0
[ 10.771448] sched_init_smp+0x2c/0x7c
[ 10.775145] kernel_init_freeable+0x138/0x2f0
[ 10.779544] kernel_init+0x10/0x120
[ 10.783064] ret_from_fork+0x10/0x18
[ 10.786671]
[ 10.788168] Freed by task 1:
[ 10.791074] __kasan_slab_free+0x114/0x220
[ 10.795209] kasan_slab_free+0x10/0x18
[ 10.798993] kfree+0x74/0x210
[ 10.801986] free_sched_groups.part.1+0x74/0xf0
[ 10.806562] destroy_sched_domain+0x50/0x118
[ 10.810874] cpu_attach_domain+0x140/0x450
[ 10.815009] build_sched_domains+0x1750/0x1820
[ 10.819496] sched_init_domains+0x8c/0xb0
[ 10.823544] sched_init_smp+0x2c/0x7c
[ 10.827241] kernel_init_freeable+0x138/0x2f0
[ 10.831639] kernel_init+0x10/0x120
[ 10.835159] ret_from_fork+0x10/0x18
[ 10.838766]
[ 10.840264] The buggy address belongs to the object at ffff801db4668000
[ 10.840264] which belongs to the cache kmalloc-128 of size 128
[ 10.852926] The buggy address is located 0 bytes inside of
[ 10.852926] 128-byte region [ffff801db4668000, ffff801db4668080)
[ 10.864619] The buggy address belongs to the page:
[ 10.869459] page:ffff7e0076d19a00 count:1 mapcount:0
mapping:ffff801dbac0fc00 index:0xffff801db4669f00 compound_mapcount: 0
[ 10.880717] flags: 0x1fffc00000008100(slab|head)
[ 10.885383] raw: 1fffc00000008100 ffff7e0076d19888 ffff7e0076d19a88
ffff801dbac0fc00
[ 10.893211] raw: ffff801db4669f00 0000000000200001 00000001ffffffff
0000000000000000
[ 10.901037] page dumped because: kasan: bad access detected
[ 10.906664]
[ 10.908162] Memory state around the buggy address:
[ 10.913001] ffff801db4667f00: fc fc fc fc fc fc fc fc fc fc fc fc fc
fc fc fc
[ 10.920300] ffff801db4667f80: fc fc fc fc fc fc fc fc fc fc fc fc fc
fc fc fc
[ 10.927599] >ffff801db4668000: fb fb fb fb fb fb fb fb fb fb fb fb fb
fb fb fb
[ 10.934897] ^
[ 10.938153] ffff801db4668080: fc fc fc fc fc fc fc fc fc fc fc fc fc
fc fc fc
[ 10.945453] ffff801db4668100: fc fc fc fc fc fc fc fc fc fc fc fc fc
fc fc fc
[ 10.952752]
==================================================================
[ 10.960050] Disabling lock debugging due to kernel taint
[ 10.982852] Unable to handle kernel NULL pointer dereference at
virtual address 0000000000000000
[ 10.991738] Mem abort info:
[ 10.994554] ESR = 0x96000004
[ 10.997641] Exception class = DABT (current EL), IL = 32 bits
[ 11.003626] SET = 0, FnV = 0
[ 11.006706] EA = 0, S1PTW = 0
[ 11.009878] Data abort info:
[ 11.012782] ISV = 0, ISS = 0x00000004
[ 11.016659] CM = 0, WnR = 0
[ 11.019651] [0000000000000000] user address but active_mm is swapper
[ 11.026078] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 11.031707] Modules linked in:
[ 11.034791] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G B
4.19.0-rc6-00018-gf417e94-dirty #945
[ 11.044550] Hardware name: Hisilicon Hip07 D05 Development Board (DT)
[ 11.051058] pstate: 40000005 (nZcv daif -PAN -UAO)
[ 11.055898] pc : free_sched_groups.part.1+0x4c/0xf0
[ 11.060825] lr : free_sched_groups.part.1+0x4c/0xf0
[ 11.065750] sp : ffff801db5d4fb60
[ 11.069093] x29: ffff801db5d4fb60 x28: 00000000ffff0000
[ 11.074462] x27: ffff2000094ad000 x26: ffff801db4b34000
[ 11.079830] x25: ffff200008c7e080 x24: 0000000000000001
[ 11.085197] x23: ffff200008c7e0c0 x22: ffff801db45b9810
[ 11.090565] x21: ffff801db45c8000 x20: 0000000000000000
[ 11.095932] x19: 0000000000000000 x18: 0000000034d5d91d
[ 11.101300] x17: 00000000b44797ff x16: 0000000000000000
[ 11.106667] x15: 0000000000000000 x14: 3d6769726f5f6773
[ 11.112035] x13: 2030303230633534 x12: 6264313038666666
[ 11.117403] x11: 6678303d6367733e x10: 2d677320313d6367
[ 11.122770] x9 : 735f656572662073 x8 : 3038633534626431
[ 11.128138] x7 : 3038666666667830 x6 : 0000000000000000
[ 11.133505] x5 : ffff2000094ad6c8 x4 : 0000000000000000
[ 11.138872] x3 : ffff200008154bf4 x2 : 0000000000000000
[ 11.144239] x1 : 176366c3c3548000 x0 : 0000000000000000
[ 11.149608] Process swapper/0 (pid: 1, stack limit = 0x(____ptrval____))
[ 11.156379] Call trace:
[ 11.158845] free_sched_groups.part.1+0x4c/0xf0
[ 11.163421] destroy_sched_domain+0x50/0x118
[ 11.167732] cpu_attach_domain+0x140/0x450
[ 11.171868] build_sched_domains+0x1750/0x1820
[ 11.176356] sched_init_domains+0x8c/0xb0
[ 11.180404] sched_init_smp+0x2c/0x7c
[ 11.184100] kernel_init_freeable+0x138/0x2f0
[ 11.188499] kernel_init+0x10/0x120
[ 11.192020] ret_from_fork+0x10/0x18
[ 11.195629] Code: eb15029f 54000200 aa1303e0 94059f92 (f9400274)
[ 11.201867] ---[ end trace 794769e2506810a5 ]---
[ 11.206558] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
[ 11.206558]
[ 11.215809] SMP: stopping secondary CPUs
[ 11.219795] ---[ end Kernel panic - not syncing: Attempted to kill
init! exitcode=0x0000000b
[ 11.219795] ]---
The KASAN bug report is essentially the same with and without this change.
I did not try the full patch, and I guess the problem would go away
after the full patch is applied, but you may want to know the issue still..
Thanks,
John
> if (!sched_domains_numa_distance)
> return;
>
> @@ -1353,39 +1369,22 @@ void sched_init_numa(void)
> * node_distance(i,j) in order to avoid cubic time.
> */
> next_distance = curr_distance;
> +
> for (i = 0; i < nr_node_ids; i++) {
> for (j = 0; j < nr_node_ids; j++) {
> - for (k = 0; k < nr_node_ids; k++) {
> - int distance = node_distance(i, k);
> -
> - if (distance > curr_distance &&
> - (distance < next_distance ||
> - next_distance == curr_distance))
> - next_distance = distance;
> -
> - /*
> - * While not a strong assumption it would be nice to know
> - * about cases where if node A is connected to B, B is not
> - * equally connected to A.
> - */
> - if (sched_debug() && node_distance(k, i) != distance)
> - sched_numa_warn("Node-distance not symmetric");
> -
> - if (sched_debug() && i && !find_numa_distance(distance))
> - sched_numa_warn("Node-0 not representative");
> - }
> - if (next_distance != curr_distance) {
> - sched_domains_numa_distance[level++] = next_distance;
> - sched_domains_numa_levels = level;
> - curr_distance = next_distance;
> - } else break;
> - }
> + int distance = node_distance(0, j);
>
> - /*
> - * In case of sched_debug() we verify the above assumption.
> - */
> - if (!sched_debug())
> - break;
> + if (distance > curr_distance &&
> + (distance < next_distance ||
> + next_distance == curr_distance))
> + next_distance = distance;
> +
> + }
> + if (next_distance != curr_distance) {
> + sched_domains_numa_distance[level++] = next_distance;
> + sched_domains_numa_levels = level;
> + curr_distance = next_distance;
> + } else break;
> }
>
> /*
> @@ -1428,7 +1427,24 @@ void sched_init_numa(void)
> sched_domains_numa_masks[i][j] = mask;
>
> for_each_node(k) {
> - if (node_distance(j, k) > sched_domains_numa_distance[i])
> + int distance = node_distance(j, k);
> +
> + if (!i && j != k) {
> + sched_numa_warn("Non-trace locality\n");
> + goto fail;
> + }
> +
> + if (distance > curr_distance) {
> + sched_numa_warn("Node-0 not complete\n");
> + goto fail;
> + }
> +
> + if (distance != node_distance(k, j)) {
> + sched_numa_warn("Non symmetric\n");
> + goto fail;
> + }
> +
> + if (distance > sched_domains_numa_distance[i])
> continue;
>
> cpumask_or(mask, mask, cpumask_of_node(k));
> @@ -1437,9 +1453,10 @@ void sched_init_numa(void)
> }
>
> /* Compute default topology size */
> - for (i = 0; sched_domain_topology[i].mask; i++);
> + for (k = 0; sched_domain_topology[k].mask; k++)
> + ;
>
> - tl = kzalloc((i + level + 1) *
> + tl = kzalloc((k + level + 1) *
> sizeof(struct sched_domain_topology_level), GFP_KERNEL);
> if (!tl)
> return;
> @@ -1447,7 +1464,7 @@ void sched_init_numa(void)
> /*
> * Copy the default topology bits..
> */
> - for (i = 0; sched_domain_topology[i].mask; i++)
> + for (i = 0; i < k; i++)
> tl[i] = sched_domain_topology[i];
>
> /*
> @@ -1478,6 +1495,31 @@ void sched_init_numa(void)
> sched_max_numa_distance = sched_domains_numa_distance[level - 1];
>
> init_numa_topology_type();
> +
> + return;
> +
> +fail:
> + /* Compute default topology size */
> + for (k = 0; sched_domain_topology[k].mask; k++)
> + ;
> +
> + tl = kzalloc((k + 2) *
> + sizeof(struct sched_domain_topology_level), GFP_KERNEL);
> + if (!tl)
> + return;
> +
> + /*
> + * Copy the default topology bits..
> + */
> + for (i = 0; i < k; i++)
> + tl[i] = sched_domain_topology[i];
> +
> + tl[i] = (struct sched_domain_topology_level){
> + .mask = cpu_system_mask,
> + SD_INIT_NAME(SYSTEM)
> + };
> +
> + sched_domain_topology = tl;
> }
>
> void sched_domains_numa_masks_set(unsigned int cpu)
>
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
> .
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-11-02 12:08 ` John Garry
@ 2018-11-02 12:19 ` Peter Zijlstra
2018-11-07 18:42 ` John Garry
0 siblings, 1 reply; 13+ messages in thread
From: Peter Zijlstra @ 2018-11-02 12:19 UTC (permalink / raw)
To: John Garry
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
Linuxarm, linux-kernel, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On Fri, Nov 02, 2018 at 12:08:53PM +0000, John Garry wrote:
> > @@ -1337,7 +1353,7 @@ void sched_init_numa(void)
> > int level = 0;
> > int i, j, k;
> >
> > - sched_domains_numa_distance = kzalloc(sizeof(int) * nr_node_ids, GFP_KERNEL);
> > + sched_domains_numa_distance = kzalloc(sizeof(int) * (nr_node_ids + 1), GFP_KERNEL);
>
> Ahhhh, unfortuately this change in isolation is not fixing the crash for me.
> I'm still crashing in the same place repeatedly.
Hurm.. sadness, means my reproduces doesn't reproduce quite enough :/
I'll see if I can somehow get something better going.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Crash report: Broken NUMA distance map causes crash on arm64 system
2018-11-02 12:19 ` Peter Zijlstra
@ 2018-11-07 18:42 ` John Garry
0 siblings, 0 replies; 13+ messages in thread
From: John Garry @ 2018-11-07 18:42 UTC (permalink / raw)
To: Peter Zijlstra
Cc: devicetree, Anshuman Khandual, Catalin Marinas, Will Deacon,
Linuxarm, linux-kernel, Rob Herring, Frank Rowand, Ingo Molnar,
linux-arm-kernel
On 02/11/2018 12:19, Peter Zijlstra wrote:
> On Fri, Nov 02, 2018 at 12:08:53PM +0000, John Garry wrote:
>>> @@ -1337,7 +1353,7 @@ void sched_init_numa(void)
>>> int level = 0;
>>> int i, j, k;
>>>
>>> - sched_domains_numa_distance = kzalloc(sizeof(int) * nr_node_ids, GFP_KERNEL);
>>> + sched_domains_numa_distance = kzalloc(sizeof(int) * (nr_node_ids + 1), GFP_KERNEL);
>>
>> Ahhhh, unfortuately this change in isolation is not fixing the crash for me.
>> I'm still crashing in the same place repeatedly.
>
> Hurm.. sadness, means my reproduces doesn't reproduce quite enough :/
> I'll see if I can somehow get something better going.
Quick update: As expected, v4.20-rc1 has the same issue.
I did notice that if I set the .flags and .sd_flags members for the NUMA
identity sched domain topology level - as done for other domains:
--- a/kernel/sched/topology.c
+++ b/kernel/sched/topology.c
@@ -1455,6 +1455,8 @@ void sched_init_numa(void)
*/
tl[i++] = (struct sched_domain_topology_level){
.mask = sd_numa_mask,
+ .sd_flags = cpu_numa_flags,
+ .flags = SDTL_OVERLAP,
.numa_level = 0,
SD_INIT_NAME(NODE)
};
... then no crash. I doubt that this is right ... but at least a fuller
log to analyse (which may be useless, considering the previous point):
[ 0.000000] Booting Linux on physical CPU 0x0000010000 [0x410fd082]
[ 0.000000] Linux version 4.20.0-rc1-00003-g68cb15521541-dirty
(johnpgarry@johnpgarry-ThinkCentre-M93p) (gcc version 7.3.1 20180425
[linaro-7.3-2018.05-rc1 revision
38aec9a676236eaa42ca03ccb3a6c1dd0182c29f] (Linaro GCC 7.3-2018.05-rc1))
#3498 SMP PREEMPT Wed Nov 7 18:28:48 GMT 2018
[ 0.000000] Machine model: Hisilicon Hip07 D05 Development Board
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: EFI v2.60 by EDK II
[ 0.000000] efi: SMBIOS=0x3f050000 SMBIOS 3.0=0x39b00000
ACPI=0x39bd0000 ACPI 2.0=0x39bd0014 MEMATTR=0x3cc99018
MEMRESERVE=0x31902e98
[ 0.000000] cannot reserve crashkernel: region overlaps reserved memory
[ 0.000000] cma: Reserved 32 MiB at 0x000000003d000000
[ 0.000000] earlycon: pl11 at MMIO32 0x00000000602b0000 (options
'115200n8')
[ 0.000000] printk: bootconsole [pl11] enabled
[ 0.000000] OF: NUMA: parsing numa-distance-map-v1
[ 0.000000] NUMA: Warning: invalid distance parameter, from=3 to=3
distance=15
[ 0.000000] NUMA: NODE_DATA [mem 0x1ffbffe840-0x1ffbffffff]
[ 0.000000] NUMA: Initmem setup node 1 [<memory-less node>]
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbfc5840-0x41ffbfc6fff]
[ 0.000000] NUMA: NODE_DATA(1) on node 2
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbfc4080-0x41ffbfc583f]
[ 0.000000] NUMA: Initmem setup node 3 [<memory-less node>]
[ 0.000000] NUMA: NODE_DATA [mem 0x41ffbfc28c0-0x41ffbfc407f]
[ 0.000000] NUMA: NODE_DATA(3) on node 2
[ 0.000000] Zone ranges:
[ 0.000000] DMA32 [mem 0x0000000000000000-0x00000000ffffffff]
[ 0.000000] Normal [mem 0x0000000100000000-0x0000041ffbffffff]
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x0000000000000000-0x0000000031902fff]
[ 0.000000] node 0: [mem 0x0000000031903000-0x0000000031906fff]
[ 0.000000] node 0: [mem 0x0000000031907000-0x000000003999ffff]
[ 0.000000] node 0: [mem 0x00000000399a0000-0x0000000039a3ffff]
[ 0.000000] node 0: [mem 0x0000000039a40000-0x0000000039a8ffff]
[ 0.000000] node 0: [mem 0x0000000039a90000-0x0000000039a9ffff]
[ 0.000000] node 0: [mem 0x0000000039aa0000-0x0000000039aeffff]
[ 0.000000] node 0: [mem 0x0000000039af0000-0x0000000039bbffff]
[ 0.000000] node 0: [mem 0x0000000039bc0000-0x0000000039bdffff]
[ 0.000000] node 0: [mem 0x0000000039be0000-0x000000003a05ffff]
[ 0.000000] node 0: [mem 0x000000003a060000-0x000000003f04ffff]
[ 0.000000] node 0: [mem 0x000000003f050000-0x000000003f07ffff]
[ 0.000000] node 0: [mem 0x000000003f080000-0x000000003fbfffff]
[ 0.000000] node 0: [mem 0x0000001040000000-0x0000001ffbffffff]
[ 0.000000] node 2: [mem 0x0000041000000000-0x0000041ffbffffff]
[ 0.000000] Zeroed struct page in unavailable ranges: 148 pages
[ 0.000000] Initmem setup node 0 [mem
0x0000000000000000-0x0000001ffbffffff]
[ 0.000000] On node 0 totalpages: 16759808
[ 0.000000] DMA32 zone: 4080 pages used for memmap
[ 0.000000] DMA32 zone: 0 pages reserved
[ 0.000000] DMA32 zone: 261120 pages, LIFO batch:63
[ 0.000000] Normal zone: 257792 pages used for memmap
[ 0.000000] Normal zone: 16498688 pages, LIFO batch:63
[ 0.000000] Could not find start_pfn for node 1
[ 0.000000] Initmem setup node 1 [mem
0x0000000000000000-0x0000000000000000]
[ 0.000000] On node 1 totalpages: 0
[ 0.000000] Initmem setup node 2 [mem
0x0000041000000000-0x0000041ffbffffff]
[ 0.000000] On node 2 totalpages: 16760832
[ 0.000000] Normal zone: 261888 pages used for memmap
[ 0.000000] Normal zone: 16760832 pages, LIFO batch:63
[ 0.000000] Could not find start_pfn for node 3
[ 0.000000] Initmem setup node 3 [mem
0x0000000000000000-0x0000000000000000]
[ 0.000000] On node 3 totalpages: 0
[ 0.000000] kasan: KernelAddressSanitizer initialized
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.0 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: MIGRATE_INFO_TYPE not supported.
[ 0.000000] psci: SMC Calling Convention v1.0
[ 0.000000] random: get_random_bytes called from
start_kernel+0x10c/0x514 with crng_init=0
[ 0.000000] percpu: Embedded 29 pages/cpu @(____ptrval____) s81752
r8192 d28840 u118784
[ 0.000000] pcpu-alloc: s81752 r8192 d28840 u118784 alloc=29*4096
[ 0.000000] pcpu-alloc: [0] 00 [0] 01 [0] 02 [0] 03 [0] 04 [0] 05 [0]
06 [0] 07
[ 0.000000] pcpu-alloc: [0] 08 [0] 09 [0] 10 [0] 11 [0] 12 [0] 13 [0]
14 [0] 15
[ 0.000000] pcpu-alloc: [1] 16 [1] 17 [1] 18 [1] 19 [1] 20 [1] 21 [1]
22 [1] 23
[ 0.000000] pcpu-alloc: [1] 24 [1] 25 [1] 26 [1] 27 [1] 28 [1] 29 [1]
30 [1] 31
[ 0.000000] pcpu-alloc: [2] 32 [2] 33 [2] 34 [2] 35 [2] 36 [2] 37 [2]
38 [2] 39
[ 0.000000] pcpu-alloc: [2] 40 [2] 41 [2] 42 [2] 43 [2] 44 [2] 45 [2]
46 [2] 47
[ 0.000000] pcpu-alloc: [3] 48 [3] 49 [3] 50 [3] 51 [3] 52 [3] 53 [3]
54 [3] 55
[ 0.000000] pcpu-alloc: [3] 56 [3] 57 [3] 58 [3] 59 [3] 60 [3] 61 [3]
62 [3] 63
[ 0.000000] Detected PIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for EL2 vector hardening
[ 0.000000] CPU features: detected: Kernel page table isolation (KPTI)
[ 0.000000] Built 4 zonelists, mobility grouping on. Total pages:
32996880
[ 0.000000] Policy zone: Normal
[ 0.000000] Kernel command line: BOOT_IMAGE=/john/Image rdinit=/init
crashkernel=256M@32M console=ttyAMA0,115200 acpi=off pcie_aspm=off
earlycon sched_debug loglevel=7
[ 0.000000] PCIe ASPM is disabled
[ 0.000000] printk: log_buf_len individual max cpu contribution: 4096
bytes
[ 0.000000] printk: log_buf_len total cpu_extra contributions: 258048
bytes
[ 0.000000] printk: log_buf_len min size: 131072 bytes
[ 0.000000] printk: log_buf_len: 524288 bytes
[ 0.000000] printk: early log buf free: 125080(95%)
[ 0.000000] software IO TLB: mapped [mem 0x359a0000-0x399a0000] (64MB)
[ 0.000000] Memory: 114779408K/134082560K available (9532K kernel
code, 6780K rwdata, 6804K rodata, 1408K init, 8717K bss, 19270384K
reserved, 32768K cma-reserved)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=64, Nodes=4
[ 0.000000] rcu: Preemptible hierarchical RCU implementation.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay
is 25 jiffies.
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] GICv3: GIC: Using split EOI/Deactivate mode
[ 0.000000] GICv3: Distributor has no Range Selector support
[ 0.000000] GICv3: VLPI support, direct LPI support
[ 0.000000] GICv3: CPU0: found redistributor 10000 region
0:0x000000004d100000
[ 0.000000] ITS [mem 0x4c000000-0x4c03ffff]
[ 0.000000] ITS@0x000000004c000000: Using ITS number 0
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000000004c000000: allocated 524288 Devices
@1dba400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000000004c000000: allocated 65536 Virtual CPUs
@1dbac80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000000004c000000: allocated 512 Interrupt
Collections @1dbac39000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x6c000000-0x6c03ffff]
[ 0.000000] ITS@0x000000006c000000: Using ITS number 1
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000000006c000000: allocated 524288 Devices
@1dba000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000000006c000000: allocated 65536 Virtual CPUs
@1dbad00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000000006c000000: allocated 512 Interrupt
Collections @1dbac3a000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0xc6000000-0xc603ffff]
[ 0.000000] ITS@0x00000000c6000000: Using ITS number 2
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000000c6000000: allocated 524288 Devices
@1db9c00000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000000c6000000: allocated 65536 Virtual CPUs
@1dbad80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000000c6000000: allocated 512 Interrupt
Collections @1dbac3c000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x8c6000000-0x8c603ffff]
[ 0.000000] ITS@0x00000008c6000000: Using ITS number 3
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000008c6000000: allocated 524288 Devices
@1db9800000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000008c6000000: allocated 65536 Virtual CPUs
@1dbae00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000008c6000000: allocated 512 Interrupt
Collections @1dbac3d000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x4004c000000-0x4004c03ffff]
[ 0.000000] ITS@0x000004004c000000: Using ITS number 4
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000004004c000000: allocated 524288 Devices
@1db9400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000004004c000000: allocated 65536 Virtual CPUs
@1dbaf00000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000004004c000000: allocated 512 Interrupt
Collections @1dbae90000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x4006c000000-0x4006c03ffff]
[ 0.000000] ITS@0x000004006c000000: Using ITS number 5
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x000004006c000000: allocated 524288 Devices
@1db9000000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x000004006c000000: allocated 65536 Virtual CPUs
@1dbaf80000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x000004006c000000: allocated 512 Interrupt
Collections @1dbae92000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x400c6000000-0x400c603ffff]
[ 0.000000] ITS@0x00000400c6000000: Using ITS number 6
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000400c6000000: allocated 524288 Devices
@1db8c00000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000400c6000000: allocated 65536 Virtual CPUs
@1db8800000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000400c6000000: allocated 512 Interrupt
Collections @1dbae93000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS [mem 0x408c6000000-0x408c603ffff]
[ 0.000000] ITS@0x00000408c6000000: Using ITS number 7
[ 0.000000] GIC: enabling workaround for ITS: Hip07 erratum 161600802
[ 0.000000] ITS@0x00000408c6000000: allocated 524288 Devices
@1db8400000 (flat, esz 8, psz 16K, shr 1)
[ 0.000000] ITS@0x00000408c6000000: allocated 65536 Virtual CPUs
@1db8880000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] ITS@0x00000408c6000000: allocated 512 Interrupt
Collections @1dbae94000 (flat, esz 8, psz 4K, shr 1)
[ 0.000000] GICv3: using LPI property table @0x0000001dbaed0000
[ 0.000000] ITS: Using DirectLPI for VPE invalidation
[ 0.000000] ITS: Enabling GICv4 support
[ 0.000000] GICv3: CPU0: using allocated LPI pending table
@0x0000001dbaee0000
[ 0.000000] arch_timer: cp15 timer(s) running at 50.00MHz (phys).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff
max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
[ 0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps
every 4398046511100ns
[ 0.008704] Console: colour dummy device 80x25
[ 0.013312] mempolicy: Enabling automatic NUMA balancing. Configure
with numa_balancing= or the kernel.numa_balancing sysctl
[ 0.024688] Calibrating delay loop (skipped), value calculated using
timer frequency.. 100.00 BogoMIPS (lpj=200000)
[ 0.035246] pid_max: default: 65536 minimum: 512
[ 0.040090] LSM: Security Framework initializing
[ 0.075362] Dentry cache hash table entries: 8388608 (order: 14,
67108864 bytes)
[ 0.097804] Inode-cache hash table entries: 4194304 (order: 13,
33554432 bytes)
[ 0.105781] Mount-cache hash table entries: 131072 (order: 8, 1048576
bytes)
[ 0.113382] Mountpoint-cache hash table entries: 131072 (order: 8,
1048576 bytes)
[ 0.181076] ASID allocator initialised with 32768 entries
[ 0.194552] rcu: Hierarchical SRCU implementation.
[ 0.207491] Platform MSI: interrupt-controller@4c000000 domain created
[ 0.214116] Platform MSI: interrupt-controller@6c000000 domain created
[ 0.220732] Platform MSI: interrupt-controller@c6000000 domain created
[ 0.227348] Platform MSI: interrupt-controller@8,c6000000 domain created
[ 0.234147] Platform MSI: interrupt-controller@400,4c000000 domain
created
[ 0.241114] Platform MSI: interrupt-controller@400,6c000000 domain
created
[ 0.248079] Platform MSI: interrupt-controller@400,c6000000 domain
created
[ 0.255048] Platform MSI: interrupt-controller@408,c6000000 domain
created
[ 0.262092] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@4c000000 domain created
[ 0.270994] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@6c000000 domain created
[ 0.279895] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@c6000000 domain created
[ 0.288795] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@8,c6000000 domain
created
[ 0.297872] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,4c000000 domain
created
[ 0.307124] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,6c000000 domain
created
[ 0.316376] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@400,c6000000 domain
created
[ 0.325628] PCI/MSI:
/interrupt-controller@4d000000/interrupt-controller@408,c6000000 domain
created
[ 0.336346] Remapping and enabling EFI services.
[ 0.349031] smp: Bringing up secondary CPUs ...
[ 0.436257] Detected PIPT I-cache on CPU1
[ 0.436274] GICv3: CPU1: found redistributor 10001 region
0:0x000000004d140000
[ 0.436295] GICv3: CPU1: using allocated LPI pending table
@0x0000001dbaef0000
[ 0.436418] CPU1: Booted secondary processor 0x0000010001 [0x410fd082]
[ 0.518922] Detected PIPT I-cache on CPU2
[ 0.518933] GICv3: CPU2: found redistributor 10002 region
0:0x000000004d180000
[ 0.518946] GICv3: CPU2: using allocated LPI pending table
@0x0000001db8900000
[ 0.519054] CPU2: Booted secondary processor 0x0000010002 [0x410fd082]
[ 0.601589] Detected PIPT I-cache on CPU3
[ 0.601601] GICv3: CPU3: found redistributor 10003 region
0:0x000000004d1c0000
[ 0.601614] GICv3: CPU3: using allocated LPI pending table
@0x0000001db8910000
[ 0.601721] CPU3: Booted secondary processor 0x0000010003 [0x410fd082]
[ 0.684259] Detected PIPT I-cache on CPU4
[ 0.684272] GICv3: CPU4: found redistributor 10100 region
0:0x000000004d200000
[ 0.684287] GICv3: CPU4: using allocated LPI pending table
@0x0000001db8920000
[ 0.684394] CPU4: Booted secondary processor 0x0000010100 [0x410fd082]
[ 0.766926] Detected PIPT I-cache on CPU5
[ 0.766938] GICv3: CPU5: found redistributor 10101 region
0:0x000000004d240000
[ 0.766950] GICv3: CPU5: using allocated LPI pending table
@0x0000001db8930000
[ 0.767059] CPU5: Booted secondary processor 0x0000010101 [0x410fd082]
[ 0.849594] Detected PIPT I-cache on CPU6
[ 0.849606] GICv3: CPU6: found redistributor 10102 region
0:0x000000004d280000
[ 0.849619] GICv3: CPU6: using allocated LPI pending table
@0x0000001db8940000
[ 0.849724] CPU6: Booted secondary processor 0x0000010102 [0x410fd082]
[ 0.932265] Detected PIPT I-cache on CPU7
[ 0.932277] GICv3: CPU7: found redistributor 10103 region
0:0x000000004d2c0000
[ 0.932290] GICv3: CPU7: using allocated LPI pending table
@0x0000001db8950000
[ 0.932398] CPU7: Booted secondary processor 0x0000010103 [0x410fd082]
[ 1.014937] Detected PIPT I-cache on CPU8
[ 1.014951] GICv3: CPU8: found redistributor 10200 region
0:0x000000004d300000
[ 1.014966] GICv3: CPU8: using allocated LPI pending table
@0x0000001db8960000
[ 1.015075] CPU8: Booted secondary processor 0x0000010200 [0x410fd082]
[ 1.097605] Detected PIPT I-cache on CPU9
[ 1.097617] GICv3: CPU9: found redistributor 10201 region
0:0x000000004d340000
[ 1.097630] GICv3: CPU9: using allocated LPI pending table
@0x0000001db8970000
[ 1.097736] CPU9: Booted secondary processor 0x0000010201 [0x410fd082]
[ 1.180275] Detected PIPT I-cache on CPU10
[ 1.180288] GICv3: CPU10: found redistributor 10202 region
0:0x000000004d380000
[ 1.180301] GICv3: CPU10: using allocated LPI pending table
@0x0000001db8980000
[ 1.180407] CPU10: Booted secondary processor 0x0000010202 [0x410fd082]
[ 1.262946] Detected PIPT I-cache on CPU11
[ 1.262958] GICv3: CPU11: found redistributor 10203 region
0:0x000000004d3c0000
[ 1.262972] GICv3: CPU11: using allocated LPI pending table
@0x0000001db8990000
[ 1.263079] CPU11: Booted secondary processor 0x0000010203 [0x410fd082]
[ 1.345617] Detected PIPT I-cache on CPU12
[ 1.345632] GICv3: CPU12: found redistributor 10300 region
0:0x000000004d400000
[ 1.345648] GICv3: CPU12: using allocated LPI pending table
@0x0000001db89a0000
[ 1.345756] CPU12: Booted secondary processor 0x0000010300 [0x410fd082]
[ 1.428285] Detected PIPT I-cache on CPU13
[ 1.428297] GICv3: CPU13: found redistributor 10301 region
0:0x000000004d440000
[ 1.428311] GICv3: CPU13: using allocated LPI pending table
@0x0000001db89b0000
[ 1.428419] CPU13: Booted secondary processor 0x0000010301 [0x410fd082]
[ 1.510954] Detected PIPT I-cache on CPU14
[ 1.510967] GICv3: CPU14: found redistributor 10302 region
0:0x000000004d480000
[ 1.510981] GICv3: CPU14: using allocated LPI pending table
@0x0000001db89c0000
[ 1.511089] CPU14: Booted secondary processor 0x0000010302 [0x410fd082]
[ 1.593637] Detected PIPT I-cache on CPU15
[ 1.593650] GICv3: CPU15: found redistributor 10303 region
0:0x000000004d4c0000
[ 1.593664] GICv3: CPU15: using allocated LPI pending table
@0x0000001db89d0000
[ 1.593770] CPU15: Booted secondary processor 0x0000010303 [0x410fd082]
[ 1.675970] Detected PIPT I-cache on CPU16
[ 1.675988] GICv3: CPU16: found redistributor 30000 region
1:0x000000006d100000
[ 1.676008] GICv3: CPU16: using allocated LPI pending table
@0x0000001db89e0000
[ 1.676162] CPU16: Booted secondary processor 0x0000030000 [0x410fd082]
[ 1.758640] Detected PIPT I-cache on CPU17
[ 1.758655] GICv3: CPU17: found redistributor 30001 region
1:0x000000006d140000
[ 1.758671] GICv3: CPU17: using allocated LPI pending table
@0x0000001db89f0000
[ 1.758819] CPU17: Booted secondary processor 0x0000030001 [0x410fd082]
[ 1.841313] Detected PIPT I-cache on CPU18
[ 1.841328] GICv3: CPU18: found redistributor 30002 region
1:0x000000006d180000
[ 1.841340] GICv3: CPU18: using allocated LPI pending table
@0x0000001db8a00000
[ 1.841488] CPU18: Booted secondary processor 0x0000030002 [0x410fd082]
[ 1.923986] Detected PIPT I-cache on CPU19
[ 1.924002] GICv3: CPU19: found redistributor 30003 region
1:0x000000006d1c0000
[ 1.924013] GICv3: CPU19: using allocated LPI pending table
@0x0000001db8a10000
[ 1.924162] CPU19: Booted secondary processor 0x0000030003 [0x410fd082]
[ 2.006664] Detected PIPT I-cache on CPU20
[ 2.006683] GICv3: CPU20: found redistributor 30100 region
1:0x000000006d200000
[ 2.006699] GICv3: CPU20: using allocated LPI pending table
@0x0000001db8a20000
[ 2.006853] CPU20: Booted secondary processor 0x0000030100 [0x410fd082]
[ 2.089332] Detected PIPT I-cache on CPU21
[ 2.089348] GICv3: CPU21: found redistributor 30101 region
1:0x000000006d240000
[ 2.089359] GICv3: CPU21: using allocated LPI pending table
@0x0000001db8a30000
[ 2.089506] CPU21: Booted secondary processor 0x0000030101 [0x410fd082]
[ 2.172007] Detected PIPT I-cache on CPU22
[ 2.172023] GICv3: CPU22: found redistributor 30102 region
1:0x000000006d280000
[ 2.172034] GICv3: CPU22: using allocated LPI pending table
@0x0000001db8a40000
[ 2.172183] CPU22: Booted secondary processor 0x0000030102 [0x410fd082]
[ 2.254680] Detected PIPT I-cache on CPU23
[ 2.254695] GICv3: CPU23: found redistributor 30103 region
1:0x000000006d2c0000
[ 2.254708] GICv3: CPU23: using allocated LPI pending table
@0x0000001db8a50000
[ 2.254857] CPU23: Booted secondary processor 0x0000030103 [0x410fd082]
[ 2.337357] Detected PIPT I-cache on CPU24
[ 2.337376] GICv3: CPU24: found redistributor 30200 region
1:0x000000006d300000
[ 2.337392] GICv3: CPU24: using allocated LPI pending table
@0x0000001db8a60000
[ 2.337547] CPU24: Booted secondary processor 0x0000030200 [0x410fd082]
[ 2.420027] Detected PIPT I-cache on CPU25
[ 2.420044] GICv3: CPU25: found redistributor 30201 region
1:0x000000006d340000
[ 2.420055] GICv3: CPU25: using allocated LPI pending table
@0x0000001db8a70000
[ 2.420203] CPU25: Booted secondary processor 0x0000030201 [0x410fd082]
[ 2.502699] Detected PIPT I-cache on CPU26
[ 2.502716] GICv3: CPU26: found redistributor 30202 region
1:0x000000006d380000
[ 2.502727] GICv3: CPU26: using allocated LPI pending table
@0x0000001db8a80000
[ 2.502875] CPU26: Booted secondary processor 0x0000030202 [0x410fd082]
[ 2.585373] Detected PIPT I-cache on CPU27
[ 2.585390] GICv3: CPU27: found redistributor 30203 region
1:0x000000006d3c0000
[ 2.585402] GICv3: CPU27: using allocated LPI pending table
@0x0000001db8a90000
[ 2.585551] CPU27: Booted secondary processor 0x0000030203 [0x410fd082]
[ 2.668052] Detected PIPT I-cache on CPU28
[ 2.668072] GICv3: CPU28: found redistributor 30300 region
1:0x000000006d400000
[ 2.668088] GICv3: CPU28: using allocated LPI pending table
@0x0000001db8aa0000
[ 2.668241] CPU28: Booted secondary processor 0x0000030300 [0x410fd082]
[ 2.750722] Detected PIPT I-cache on CPU29
[ 2.750739] GICv3: CPU29: found redistributor 30301 region
1:0x000000006d440000
[ 2.750751] GICv3: CPU29: using allocated LPI pending table
@0x0000001db8ab0000
[ 2.750900] CPU29: Booted secondary processor 0x0000030301 [0x410fd082]
[ 2.825388] Detected PIPT I-cache on CPU30
[ 2.825405] GICv3: CPU30: found redistributor 30302 region
1:0x000000006d480000
[ 2.825417] GICv3: CPU30: using allocated LPI pending table
@0x0000001db8ac0000
[ 2.825567] CPU30: Booted secondary processor 0x0000030302 [0x410fd082]
[ 2.908063] Detected PIPT I-cache on CPU31
[ 2.908080] GICv3: CPU31: found redistributor 30303 region
1:0x000000006d4c0000
[ 2.908093] GICv3: CPU31: using allocated LPI pending table
@0x0000001db8ad0000
[ 2.908242] CPU31: Booted secondary processor 0x0000030303 [0x410fd082]
[ 2.990430] Detected PIPT I-cache on CPU32
[ 2.990477] GICv3: CPU32: found redistributor 50000 region
2:0x000004004d100000
[ 2.990514] GICv3: CPU32: using allocated LPI pending table
@0x0000001db8ae0000
[ 2.990675] CPU32: Booted secondary processor 0x0000050000 [0x410fd082]
[ 3.073089] Detected PIPT I-cache on CPU33
[ 3.073118] GICv3: CPU33: found redistributor 50001 region
2:0x000004004d140000
[ 3.073130] GICv3: CPU33: using allocated LPI pending table
@0x0000001db8af0000
[ 3.073239] CPU33: Booted secondary processor 0x0000050001 [0x410fd082]
[ 3.155775] Detected PIPT I-cache on CPU34
[ 3.155803] GICv3: CPU34: found redistributor 50002 region
2:0x000004004d180000
[ 3.155819] GICv3: CPU34: using allocated LPI pending table
@0x0000001db8b00000
[ 3.155928] CPU34: Booted secondary processor 0x0000050002 [0x410fd082]
[ 3.238461] Detected PIPT I-cache on CPU35
[ 3.238490] GICv3: CPU35: found redistributor 50003 region
2:0x000004004d1c0000
[ 3.238503] GICv3: CPU35: using allocated LPI pending table
@0x0000001db8b10000
[ 3.238613] CPU35: Booted secondary processor 0x0000050003 [0x410fd082]
[ 3.321152] Detected PIPT I-cache on CPU36
[ 3.321183] GICv3: CPU36: found redistributor 50100 region
2:0x000004004d200000
[ 3.321199] GICv3: CPU36: using allocated LPI pending table
@0x0000001db8b20000
[ 3.321312] CPU36: Booted secondary processor 0x0000050100 [0x410fd082]
[ 3.403837] Detected PIPT I-cache on CPU37
[ 3.403867] GICv3: CPU37: found redistributor 50101 region
2:0x000004004d240000
[ 3.403881] GICv3: CPU37: using allocated LPI pending table
@0x0000001db8b30000
[ 3.403990] CPU37: Booted secondary processor 0x0000050101 [0x410fd082]
[ 3.486525] Detected PIPT I-cache on CPU38
[ 3.486555] GICv3: CPU38: found redistributor 50102 region
2:0x000004004d280000
[ 3.486567] GICv3: CPU38: using allocated LPI pending table
@0x0000001db8b40000
[ 3.486677] CPU38: Booted secondary processor 0x0000050102 [0x410fd082]
[ 3.569212] Detected PIPT I-cache on CPU39
[ 3.569242] GICv3: CPU39: found redistributor 50103 region
2:0x000004004d2c0000
[ 3.569255] GICv3: CPU39: using allocated LPI pending table
@0x0000001db8b50000
[ 3.569365] CPU39: Booted secondary processor 0x0000050103 [0x410fd082]
[ 3.651903] Detected PIPT I-cache on CPU40
[ 3.651934] GICv3: CPU40: found redistributor 50200 region
2:0x000004004d300000
[ 3.651949] GICv3: CPU40: using allocated LPI pending table
@0x0000001db8b60000
[ 3.652061] CPU40: Booted secondary processor 0x0000050200 [0x410fd082]
[ 3.734587] Detected PIPT I-cache on CPU41
[ 3.734617] GICv3: CPU41: found redistributor 50201 region
2:0x000004004d340000
[ 3.734631] GICv3: CPU41: using allocated LPI pending table
@0x0000001db8b70000
[ 3.734742] CPU41: Booted secondary processor 0x0000050201 [0x410fd082]
[ 3.817274] Detected PIPT I-cache on CPU42
[ 3.817304] GICv3: CPU42: found redistributor 50202 region
2:0x000004004d380000
[ 3.817319] GICv3: CPU42: using allocated LPI pending table
@0x0000001db8b80000
[ 3.817431] CPU42: Booted secondary processor 0x0000050202 [0x410fd082]
[ 3.899963] Detected PIPT I-cache on CPU43
[ 3.899993] GICv3: CPU43: found redistributor 50203 region
2:0x000004004d3c0000
[ 3.900006] GICv3: CPU43: using allocated LPI pending table
@0x0000001db8b90000
[ 3.900117] CPU43: Booted secondary processor 0x0000050203 [0x410fd082]
[ 3.982652] Detected PIPT I-cache on CPU44
[ 3.982684] GICv3: CPU44: found redistributor 50300 region
2:0x000004004d400000
[ 3.982700] GICv3: CPU44: using allocated LPI pending table
@0x0000001db8ba0000
[ 3.982814] CPU44: Booted secondary processor 0x0000050300 [0x410fd082]
[ 4.065337] Detected PIPT I-cache on CPU45
[ 4.065367] GICv3: CPU45: found redistributor 50301 region
2:0x000004004d440000
[ 4.065381] GICv3: CPU45: using allocated LPI pending table
@0x0000001db8bb0000
[ 4.065492] CPU45: Booted secondary processor 0x0000050301 [0x410fd082]
[ 4.148024] Detected PIPT I-cache on CPU46
[ 4.148055] GICv3: CPU46: found redistributor 50302 region
2:0x000004004d480000
[ 4.148070] GICv3: CPU46: using allocated LPI pending table
@0x0000001db8bc0000
[ 4.148181] CPU46: Booted secondary processor 0x0000050302 [0x410fd082]
[ 4.230729] Detected PIPT I-cache on CPU47
[ 4.230760] GICv3: CPU47: found redistributor 50303 region
2:0x000004004d4c0000
[ 4.230775] GICv3: CPU47: using allocated LPI pending table
@0x0000001db8bd0000
[ 4.230885] CPU47: Booted secondary processor 0x0000050303 [0x410fd082]
[ 4.313078] Detected PIPT I-cache on CPU48
[ 4.313115] GICv3: CPU48: found redistributor 70000 region
3:0x000004006d100000
[ 4.313134] GICv3: CPU48: using allocated LPI pending table
@0x0000001db8be0000
[ 4.313291] CPU48: Booted secondary processor 0x0000070000 [0x410fd082]
[ 4.395762] Detected PIPT I-cache on CPU49
[ 4.395796] GICv3: CPU49: found redistributor 70001 region
3:0x000004006d140000
[ 4.395811] GICv3: CPU49: using allocated LPI pending table
@0x0000001db8bf0000
[ 4.395960] CPU49: Booted secondary processor 0x0000070001 [0x410fd082]
[ 4.478451] Detected PIPT I-cache on CPU50
[ 4.478484] GICv3: CPU50: found redistributor 70002 region
3:0x000004006d180000
[ 4.478508] GICv3: CPU50: using allocated LPI pending table
@0x0000001db8000000
[ 4.478659] CPU50: Booted secondary processor 0x0000070002 [0x410fd082]
[ 4.561139] Detected PIPT I-cache on CPU51
[ 4.561173] GICv3: CPU51: found redistributor 70003 region
3:0x000004006d1c0000
[ 4.561189] GICv3: CPU51: using allocated LPI pending table
@0x0000001db8010000
[ 4.561338] CPU51: Booted secondary processor 0x0000070003 [0x410fd082]
[ 4.643831] Detected PIPT I-cache on CPU52
[ 4.643869] GICv3: CPU52: found redistributor 70100 region
3:0x000004006d200000
[ 4.643889] GICv3: CPU52: using allocated LPI pending table
@0x0000001db8020000
[ 4.644046] CPU52: Booted secondary processor 0x0000070100 [0x410fd082]
[ 4.726517] Detected PIPT I-cache on CPU53
[ 4.726551] GICv3: CPU53: found redistributor 70101 region
3:0x000004006d240000
[ 4.726566] GICv3: CPU53: using allocated LPI pending table
@0x0000001db8030000
[ 4.726715] CPU53: Booted secondary processor 0x0000070101 [0x410fd082]
[ 4.809206] Detected PIPT I-cache on CPU54
[ 4.809240] GICv3: CPU54: found redistributor 70102 region
3:0x000004006d280000
[ 4.809256] GICv3: CPU54: using allocated LPI pending table
@0x0000001db8040000
[ 4.809406] CPU54: Booted secondary processor 0x0000070102 [0x410fd082]
[ 4.891894] Detected PIPT I-cache on CPU55
[ 4.891929] GICv3: CPU55: found redistributor 70103 region
3:0x000004006d2c0000
[ 4.891944] GICv3: CPU55: using allocated LPI pending table
@0x0000001db8050000
[ 4.892093] CPU55: Booted secondary processor 0x0000070103 [0x410fd082]
[ 4.974586] Detected PIPT I-cache on CPU56
[ 4.974624] GICv3: CPU56: found redistributor 70200 region
3:0x000004006d300000
[ 4.974646] GICv3: CPU56: using allocated LPI pending table
@0x0000001db8060000
[ 4.974803] CPU56: Booted secondary processor 0x0000070200 [0x410fd082]
[ 5.057271] Detected PIPT I-cache on CPU57
[ 5.057306] GICv3: CPU57: found redistributor 70201 region
3:0x000004006d340000
[ 5.057321] GICv3: CPU57: using allocated LPI pending table
@0x0000001db8070000
[ 5.057471] CPU57: Booted secondary processor 0x0000070201 [0x410fd082]
[ 5.139961] Detected PIPT I-cache on CPU58
[ 5.139996] GICv3: CPU58: found redistributor 70202 region
3:0x000004006d380000
[ 5.140011] GICv3: CPU58: using allocated LPI pending table
@0x0000001db8080000
[ 5.140162] CPU58: Booted secondary processor 0x0000070202 [0x410fd082]
[ 5.222651] Detected PIPT I-cache on CPU59
[ 5.222686] GICv3: CPU59: found redistributor 70203 region
3:0x000004006d3c0000
[ 5.222710] GICv3: CPU59: using allocated LPI pending table
@0x0000001db8090000
[ 5.222859] CPU59: Booted secondary processor 0x0000070203 [0x410fd082]
[ 5.305342] Detected PIPT I-cache on CPU60
[ 5.305380] GICv3: CPU60: found redistributor 70300 region
3:0x000004006d400000
[ 5.305402] GICv3: CPU60: using allocated LPI pending table
@0x0000001db80a0000
[ 5.305559] CPU60: Booted secondary processor 0x0000070300 [0x410fd082]
[ 5.388027] Detected PIPT I-cache on CPU61
[ 5.388062] GICv3: CPU61: found redistributor 70301 region
3:0x000004006d440000
[ 5.388086] GICv3: CPU61: using allocated LPI pending table
@0x0000001db80b0000
[ 5.388235] CPU61: Booted secondary processor 0x0000070301 [0x410fd082]
[ 5.470714] Detected PIPT I-cache on CPU62
[ 5.470750] GICv3: CPU62: found redistributor 70302 region
3:0x000004006d480000
[ 5.470766] GICv3: CPU62: using allocated LPI pending table
@0x0000001db80c0000
[ 5.470916] CPU62: Booted secondary processor 0x0000070302 [0x410fd082]
[ 5.553405] Detected PIPT I-cache on CPU63
[ 5.553440] GICv3: CPU63: found redistributor 70303 region
3:0x000004006d4c0000
[ 5.553457] GICv3: CPU63: using allocated LPI pending table
@0x0000001db80d0000
[ 5.553606] CPU63: Booted secondary processor 0x0000070303 [0x410fd082]
[ 5.553763] smp: Brought up 4 nodes, 64 CPUs
[ 7.169648] SMP: Total of 64 processors activated.
[ 7.174498] CPU features: detected: GIC system register CPU interface
[ 7.181049] CPU features: detected: 32-bit EL0 Support
[ 7.186248] CPU features: detected: CRC32 instructions
[ 10.204147] CPU: All CPU(s) started at EL2
[ 10.208466] alternatives: patching kernel code
[ 10.213364] ERROR: Node-distance not symmetric
[ 10.219353] 10 15 20 25
[ 10.222085] 15 10 25 30
[ 10.224821] 20 25 10 15
[ 10.227552] 10 10 15 10
[ 10.241326] CPU0 attaching sched-domain(s):
[ 10.241329] domain-0: span=0-3 level=MC
[ 10.241333] groups: 0:{ span=0 }, 1:{ span=1 }, 2:{ span=2 }, 3:{
span=3 }
[ 10.241344] domain-1: span=0-15 level=DIE
[ 10.241347] groups: 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096
}, 8:{ span=8-11 cap=4096 }, 12:{ span=12-15 cap=4096 }
[ 10.241361] domain-2: span=0-15 level=NODE
[ 10.241364] groups: 0:{ span=0-15 cap=16384 }
[ 10.241369] domain-3: span=0-31 level=NUMA
[ 10.241372] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.241380] domain-4: span=0-47 level=NUMA
[ 10.241383] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.241393] ERROR: groups don't span domain->span
[ 10.246153] domain-5: span=0-63 level=NUMA
[ 10.246157] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.246163] ERROR: groups don't span domain->span
[ 10.250924] CPU1 attaching sched-domain(s):
[ 10.250926] domain-0: span=0-3 level=MC
[ 10.250929] groups: 1:{ span=1 }, 2:{ span=2 }, 3:{ span=3 }, 0:{
span=0 }
[ 10.250939] domain-1: span=0-15 level=DIE
[ 10.250942] groups: 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096
}, 8:{ span=8-11 cap=4096 }, 12:{ span=12-15 cap=4096 }
[ 10.250955] domain-2: span=0-15 level=NODE
[ 10.250958] groups: 0:{ span=0-15 cap=16384 }
[ 10.250963] domain-3: span=0-31 level=NUMA
[ 10.250966] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.250974] domain-4: span=0-47 level=NUMA
[ 10.250977] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.250986] ERROR: groups don't span domain->span
[ 10.255745] domain-5: span=0-63 level=NUMA
[ 10.255749] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.255755] ERROR: groups don't span domain->span
[ 10.260512] CPU2 attaching sched-domain(s):
[ 10.260514] domain-0: span=0-3 level=MC
[ 10.260517] groups: 2:{ span=2 }, 3:{ span=3 }, 0:{ span=0 }, 1:{
span=1 }
[ 10.260527] domain-1: span=0-15 level=DIE
[ 10.260530] groups: 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096
}, 8:{ span=8-11 cap=4096 }, 12:{ span=12-15 cap=4096 }
[ 10.260542] domain-2: span=0-15 level=NODE
[ 10.260545] groups: 0:{ span=0-15 cap=16384 }
[ 10.260550] domain-3: span=0-31 level=NUMA
[ 10.260554] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.260561] domain-4: span=0-47 level=NUMA
[ 10.260565] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.260574] ERROR: groups don't span domain->span
[ 10.265329] domain-5: span=0-63 level=NUMA
[ 10.265332] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.265338] ERROR: groups don't span domain->span
[ 10.270096] CPU3 attaching sched-domain(s):
[ 10.270098] domain-0: span=0-3 level=MC
[ 10.270101] groups: 3:{ span=3 }, 0:{ span=0 }, 1:{ span=1 }, 2:{
span=2 }
[ 10.270110] domain-1: span=0-15 level=DIE
[ 10.270114] groups: 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096
}, 8:{ span=8-11 cap=4096 }, 12:{ span=12-15 cap=4096 }
[ 10.270126] domain-2: span=0-15 level=NODE
[ 10.270129] groups: 0:{ span=0-15 cap=16384 }
[ 10.270134] domain-3: span=0-31 level=NUMA
[ 10.270137] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.270145] domain-4: span=0-47 level=NUMA
[ 10.270148] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.270157] ERROR: groups don't span domain->span
[ 10.274912] domain-5: span=0-63 level=NUMA
[ 10.274916] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.274923] ERROR: groups don't span domain->span
[ 10.279679] CPU4 attaching sched-domain(s):
[ 10.279681] domain-0: span=4-7 level=MC
[ 10.279684] groups: 4:{ span=4 }, 5:{ span=5 }, 6:{ span=6 }, 7:{
span=7 }
[ 10.279694] domain-1: span=0-15 level=DIE
[ 10.279697] groups: 4:{ span=4-7 cap=4096 }, 8:{ span=8-11
cap=4096 }, 12:{ span=12-15 cap=4096 }, 0:{ span=0-3 cap=4096 }
[ 10.279710] domain-2: span=0-15 level=NODE
[ 10.279713] groups: 0:{ span=0-15 cap=16384 }
[ 10.279718] domain-3: span=0-31 level=NUMA
[ 10.279721] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.279728] domain-4: span=0-47 level=NUMA
[ 10.279731] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.279740] ERROR: groups don't span domain->span
[ 10.284495] domain-5: span=0-63 level=NUMA
[ 10.284499] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.284505] ERROR: groups don't span domain->span
[ 10.289262] CPU5 attaching sched-domain(s):
[ 10.289264] domain-0: span=4-7 level=MC
[ 10.289267] groups: 5:{ span=5 }, 6:{ span=6 }, 7:{ span=7 }, 4:{
span=4 }
[ 10.289276] domain-1: span=0-15 level=DIE
[ 10.289280] groups: 4:{ span=4-7 cap=4096 }, 8:{ span=8-11
cap=4096 }, 12:{ span=12-15 cap=4096 }, 0:{ span=0-3 cap=4096 }
[ 10.289292] domain-2: span=0-15 level=NODE
[ 10.289295] groups: 0:{ span=0-15 cap=16384 }
[ 10.289300] domain-3: span=0-31 level=NUMA
[ 10.289303] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.289311] domain-4: span=0-47 level=NUMA
[ 10.289314] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.289323] ERROR: groups don't span domain->span
[ 10.294079] domain-5: span=0-63 level=NUMA
[ 10.294082] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.294089] ERROR: groups don't span domain->span
[ 10.298846] CPU6 attaching sched-domain(s):
[ 10.298847] domain-0: span=4-7 level=MC
[ 10.298850] groups: 6:{ span=6 }, 7:{ span=7 }, 4:{ span=4 }, 5:{
span=5 }
[ 10.298860] domain-1: span=0-15 level=DIE
[ 10.298863] groups: 4:{ span=4-7 cap=4096 }, 8:{ span=8-11
cap=4096 }, 12:{ span=12-15 cap=4096 }, 0:{ span=0-3 cap=4096 }
[ 10.298875] domain-2: span=0-15 level=NODE
[ 10.298878] groups: 0:{ span=0-15 cap=16384 }
[ 10.298883] domain-3: span=0-31 level=NUMA
[ 10.298887] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.298894] domain-4: span=0-47 level=NUMA
[ 10.298897] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.298906] ERROR: groups don't span domain->span
[ 10.303662] domain-5: span=0-63 level=NUMA
[ 10.303665] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.303671] ERROR: groups don't span domain->span
[ 10.308429] CPU7 attaching sched-domain(s):
[ 10.308431] domain-0: span=4-7 level=MC
[ 10.308434] groups: 7:{ span=7 }, 4:{ span=4 }, 5:{ span=5 }, 6:{
span=6 }
[ 10.308443] domain-1: span=0-15 level=DIE
[ 10.308446] groups: 4:{ span=4-7 cap=4096 }, 8:{ span=8-11
cap=4096 }, 12:{ span=12-15 cap=4096 }, 0:{ span=0-3 cap=4096 }
[ 10.308459] domain-2: span=0-15 level=NODE
[ 10.308462] groups: 0:{ span=0-15 cap=16384 }
[ 10.308467] domain-3: span=0-31 level=NUMA
[ 10.308470] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.308477] domain-4: span=0-47 level=NUMA
[ 10.308481] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.308490] ERROR: groups don't span domain->span
[ 10.313245] domain-5: span=0-63 level=NUMA
[ 10.313248] groups: 0:{ span=0-47 mask=0-15 cap=49152 }
[ 10.313255] ERROR: groups don't span domain->span
[ 10.318012] CPU8 attaching sched-domain(s):
[ 10.318014] domain-0: span=8-11 level=MC
[ 10.318017] groups: 8:{ span=8 }, 9:{ span=9 }, 10:{ span=10 },
11:{ span=11 }
[ 10.318027] domain-1: span=0-15 level=DIE
[ 10.318030] groups: 8:{ span=8-11 cap=4096 }, 12:{ span=12-15
cap=4096 }, 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096 }
[ 10.318043] domain-2: span=0-15 level=NODE
[ 10.318046] groups: 0:{ span=0-15 cap=16384 }
[ 10.318050] domain-3: span=0-31 level=NUMA
[ 10.318053] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.318061] domain-4: span=0-47 level=NUMA
[ 10.318064] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.318073] ERROR: groups don't span domain->span
[ 10.322828] domain-5: span=0-63 level=NUMA
[ 10.322832] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.322838] ERROR: groups don't span domain->span
[ 10.327595] CPU9 attaching sched-domain(s):
[ 10.327597] domain-0: span=8-11 level=MC
[ 10.327600] groups: 9:{ span=9 }, 10:{ span=10 }, 11:{ span=11 },
8:{ span=8 }
[ 10.327610] domain-1: span=0-15 level=DIE
[ 10.327613] groups: 8:{ span=8-11 cap=4096 }, 12:{ span=12-15
cap=4096 }, 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096 }
[ 10.327625] domain-2: span=0-15 level=NODE
[ 10.327628] groups: 0:{ span=0-15 cap=16384 }
[ 10.327633] domain-3: span=0-31 level=NUMA
[ 10.327636] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.327644] domain-4: span=0-47 level=NUMA
[ 10.327647] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.327656] ERROR: groups don't span domain->span
[ 10.332411] domain-5: span=0-63 level=NUMA
[ 10.332415] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.332421] ERROR: groups don't span domain->span
[ 10.337177] CPU10 attaching sched-domain(s):
[ 10.337179] domain-0: span=8-11 level=MC
[ 10.337182] groups: 10:{ span=10 }, 11:{ span=11 }, 8:{ span=8 },
9:{ span=9 }
[ 10.337192] domain-1: span=0-15 level=DIE
[ 10.337195] groups: 8:{ span=8-11 cap=4096 }, 12:{ span=12-15
cap=4096 }, 0:{ span=0-3 cap=4096 }, 4:{ span=4-7 cap=4096 }
[ 10.337208] domain-2: span=0-15 level=NODE
[ 10.337211] groups: 0:{ span=0-15 cap=16384 }
[ 10.337216] domain-3: span=0-31 level=NUMA
[ 10.337219] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.337227] domain-4: span=0-47 level=NUMA
[ 10.337230] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.337238] ERROR: groups don't span domain->span
[ 10.341994] domain-5: span=0-63 level=NUMA
[ 10.341997] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.342003] ERROR: groups don't span domain->span
[ 10.346760] CPU11 attaching sched-domain(s):
[ 10.346762] domain-0: span=8-11 level=MC
[ 10.346765] groups: 11:{ span=11 }, 8:{ span=8 }, 9:{ span=9 },
10:{ span=10 }
[ 10.346775] domain-1: span=0-15 level=DIE
[ 10.346778] groups: 8:{ span=8-11 cap=4096 }, 12:{ span=12-15
cap=4096 }, 0:{ span=0-3 cap=4095 }, 4:{ span=4-7 cap=4096 }
[ 10.346791] domain-2: span=0-15 level=NODE
[ 10.346794] groups: 0:{ span=0-15 cap=16383 }
[ 10.346799] domain-3: span=0-31 level=NUMA
[ 10.346802] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.346809] domain-4: span=0-47 level=NUMA
[ 10.346813] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.346822] ERROR: groups don't span domain->span
[ 10.351577] domain-5: span=0-63 level=NUMA
[ 10.351581] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.351587] ERROR: groups don't span domain->span
[ 10.356344] CPU12 attaching sched-domain(s):
[ 10.356345] domain-0: span=12-15 level=MC
[ 10.356348] groups: 12:{ span=12 }, 13:{ span=13 }, 14:{ span=14 },
15:{ span=15 }
[ 10.356358] domain-1: span=0-15 level=DIE
[ 10.356361] groups: 12:{ span=12-15 cap=4096 }, 0:{ span=0-3
cap=4095 }, 4:{ span=4-7 cap=4096 }, 8:{ span=8-11 cap=4096 }
[ 10.356374] domain-2: span=0-15 level=NODE
[ 10.356377] groups: 0:{ span=0-15 cap=16383 }
[ 10.356381] domain-3: span=0-31 level=NUMA
[ 10.356385] groups: 0:{ span=0-15 cap=16384 }, 16:{ span=16-31
cap=16384 }
[ 10.356392] domain-4: span=0-47 level=NUMA
[ 10.356396] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.356404] ERROR: groups don't span domain->span
[ 10.361160] domain-5: span=0-63 level=NUMA
[ 10.361163] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.361170] ERROR: groups don't span domain->span
[ 10.365927] CPU13 attaching sched-domain(s):
[ 10.365929] domain-0: span=12-15 level=MC
[ 10.365932] groups: 13:{ span=13 }, 14:{ span=14 }, 15:{ span=15 },
12:{ span=12 }
[ 10.365941] domain-1: span=0-15 level=DIE
[ 10.365944] groups: 12:{ span=12-15 cap=4096 }, 0:{ span=0-3
cap=4095 }, 4:{ span=4-7 cap=4096 }, 8:{ span=8-11 cap=4096 }
[ 10.365957] domain-2: span=0-15 level=NODE
[ 10.365960] groups: 0:{ span=0-15 cap=16383 }
[ 10.365965] domain-3: span=0-31 level=NUMA
[ 10.365968] groups: 0:{ span=0-15 cap=16383 }, 16:{ span=16-31
cap=16384 }
[ 10.365976] domain-4: span=0-47 level=NUMA
[ 10.365979] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.365988] ERROR: groups don't span domain->span
[ 10.370743] domain-5: span=0-63 level=NUMA
[ 10.370747] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.370752] ERROR: groups don't span domain->span
[ 10.375510] CPU14 attaching sched-domain(s):
[ 10.375511] domain-0: span=12-15 level=MC
[ 10.375514] groups: 14:{ span=14 }, 15:{ span=15 }, 12:{ span=12 },
13:{ span=13 }
[ 10.375524] domain-1: span=0-15 level=DIE
[ 10.375527] groups: 12:{ span=12-15 cap=4096 }, 0:{ span=0-3
cap=4095 }, 4:{ span=4-7 cap=4096 }, 8:{ span=8-11 cap=4096 }
[ 10.375540] domain-2: span=0-15 level=NODE
[ 10.375543] groups: 0:{ span=0-15 cap=16383 }
[ 10.375548] domain-3: span=0-31 level=NUMA
[ 10.375551] groups: 0:{ span=0-15 cap=16383 }, 16:{ span=16-31
cap=16384 }
[ 10.375558] domain-4: span=0-47 level=NUMA
[ 10.375562] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.375570] ERROR: groups don't span domain->span
[ 10.380326] domain-5: span=0-63 level=NUMA
[ 10.380329] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.380336] ERROR: groups don't span domain->span
[ 10.385093] CPU15 attaching sched-domain(s):
[ 10.385095] domain-0: span=12-15 level=MC
[ 10.385098] groups: 15:{ span=15 }, 12:{ span=12 }, 13:{ span=13 },
14:{ span=14 }
[ 10.385107] domain-1: span=0-15 level=DIE
[ 10.385111] groups: 12:{ span=12-15 cap=4096 }, 0:{ span=0-3
cap=4095 }, 4:{ span=4-7 cap=4096 }, 8:{ span=8-11 cap=4096 }
[ 10.385123] domain-2: span=0-15 level=NODE
[ 10.385126] groups: 0:{ span=0-15 cap=16383 }
[ 10.385131] domain-3: span=0-31 level=NUMA
[ 10.385134] groups: 0:{ span=0-15 cap=16383 }, 16:{ span=16-31
cap=16384 }
[ 10.385142] domain-4: span=0-47 level=NUMA
[ 10.385145] groups: 0:{ span=0-31 cap=32768 }, 32:{ span=32-63
mask=32-47 cap=32768 }
[ 10.385154] ERROR: groups don't span domain->span
[ 10.389909] domain-5: span=0-63 level=NUMA
[ 10.389912] groups: 0:{ span=0-47 mask=0-15 cap=49151 }
[ 10.389918] ERROR: groups don't span domain->span
[ 10.394685] CPU16 attaching sched-domain(s):
[ 10.394687] domain-0: span=16-19 level=MC
[ 10.394690] groups: 16:{ span=16 }, 17:{ span=17 }, 18:{ span=18 },
19:{ span=19 }
[ 10.394700] domain-1: span=16-31 level=DIE
[ 10.394703] groups: 16:{ span=16-19 cap=4096 }, 20:{ span=20-23
cap=4096 }, 24:{ span=24-27 cap=4096 }, 28:{ span=28-31 cap=4096 }
[ 10.394716] domain-2: span=16-31 level=NODE
[ 10.394720] groups: 16:{ span=16-31 cap=16384 }
[ 10.394725] domain-3: span=0-31 level=NUMA
[ 10.394728] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16383 }
[ 10.394736] domain-4: span=0-47 level=NUMA
[ 10.394739] groups: 16:{ span=0-31 mask=16-31 cap=32768 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.394749] ERROR: groups don't span domain->span
[ 10.399506] domain-5: span=0-63 level=NUMA
[ 10.399509] groups: 16:{ span=0-47 mask=16-31 cap=49152 }
[ 10.399515] ERROR: groups don't span domain->span
[ 10.404276] CPU17 attaching sched-domain(s):
[ 10.404278] domain-0: span=16-19 level=MC
[ 10.404281] groups: 17:{ span=17 }, 18:{ span=18 }, 19:{ span=19 },
16:{ span=16 }
[ 10.404291] domain-1: span=16-31 level=DIE
[ 10.404294] groups: 16:{ span=16-19 cap=4096 }, 20:{ span=20-23
cap=4096 }, 24:{ span=24-27 cap=4096 }, 28:{ span=28-31 cap=4096 }
[ 10.404306] domain-2: span=16-31 level=NODE
[ 10.404310] groups: 16:{ span=16-31 cap=16384 }
[ 10.404315] domain-3: span=0-31 level=NUMA
[ 10.404318] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16383 }
[ 10.404326] domain-4: span=0-47 level=NUMA
[ 10.404329] groups: 16:{ span=0-31 mask=16-31 cap=32768 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.404339] ERROR: groups don't span domain->span
[ 10.409095] domain-5: span=0-63 level=NUMA
[ 10.409098] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.409105] ERROR: groups don't span domain->span
[ 10.413865] CPU18 attaching sched-domain(s):
[ 10.413867] domain-0: span=16-19 level=MC
[ 10.413870] groups: 18:{ span=18 }, 19:{ span=19 }, 16:{ span=16 },
17:{ span=17 }
[ 10.413880] domain-1: span=16-31 level=DIE
[ 10.413884] groups: 16:{ span=16-19 cap=4096 }, 20:{ span=20-23
cap=4096 }, 24:{ span=24-27 cap=4096 }, 28:{ span=28-31 cap=4096 }
[ 10.413896] domain-2: span=16-31 level=NODE
[ 10.413900] groups: 16:{ span=16-31 cap=16384 }
[ 10.413905] domain-3: span=0-31 level=NUMA
[ 10.413908] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16383 }
[ 10.413916] domain-4: span=0-47 level=NUMA
[ 10.413919] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.413929] ERROR: groups don't span domain->span
[ 10.418685] domain-5: span=0-63 level=NUMA
[ 10.418689] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.418695] ERROR: groups don't span domain->span
[ 10.423455] CPU19 attaching sched-domain(s):
[ 10.423457] domain-0: span=16-19 level=MC
[ 10.423460] groups: 19:{ span=19 }, 16:{ span=16 cap=1023 }, 17:{
span=17 }, 18:{ span=18 }
[ 10.423470] domain-1: span=16-31 level=DIE
[ 10.423473] groups: 16:{ span=16-19 cap=4096 }, 20:{ span=20-23
cap=4096 }, 24:{ span=24-27 cap=4096 }, 28:{ span=28-31 cap=4096 }
[ 10.423486] domain-2: span=16-31 level=NODE
[ 10.423490] groups: 16:{ span=16-31 cap=16384 }
[ 10.423495] domain-3: span=0-31 level=NUMA
[ 10.423498] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16383 }
[ 10.423506] domain-4: span=0-47 level=NUMA
[ 10.423509] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.423519] ERROR: groups don't span domain->span
[ 10.428275] domain-5: span=0-63 level=NUMA
[ 10.428278] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.428284] ERROR: groups don't span domain->span
[ 10.433045] CPU20 attaching sched-domain(s):
[ 10.433047] domain-0: span=20-23 level=MC
[ 10.433050] groups: 20:{ span=20 }, 21:{ span=21 }, 22:{ span=22 },
23:{ span=23 }
[ 10.433060] domain-1: span=16-31 level=DIE
[ 10.433063] groups: 20:{ span=20-23 cap=4096 }, 24:{ span=24-27
cap=4096 }, 28:{ span=28-31 cap=4096 }, 16:{ span=16-19 cap=4096 }
[ 10.433076] domain-2: span=16-31 level=NODE
[ 10.433079] groups: 16:{ span=16-31 cap=16384 }
[ 10.433084] domain-3: span=0-31 level=NUMA
[ 10.433087] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16383 }
[ 10.433095] domain-4: span=0-47 level=NUMA
[ 10.433098] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.433108] ERROR: groups don't span domain->span
[ 10.437864] domain-5: span=0-63 level=NUMA
[ 10.437868] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.437874] ERROR: groups don't span domain->span
[ 10.442634] CPU21 attaching sched-domain(s):
[ 10.442636] domain-0: span=20-23 level=MC
[ 10.442639] groups: 21:{ span=21 }, 22:{ span=22 }, 23:{ span=23 },
20:{ span=20 }
[ 10.442649] domain-1: span=16-31 level=DIE
[ 10.442651] groups: 20:{ span=20-23 cap=4096 }, 24:{ span=24-27
cap=4096 }, 28:{ span=28-31 cap=4096 }, 16:{ span=16-19 cap=4096 }
[ 10.442664] domain-2: span=16-31 level=NODE
[ 10.442667] groups: 16:{ span=16-31 cap=16384 }
[ 10.442672] domain-3: span=0-31 level=NUMA
[ 10.442676] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16382 }
[ 10.442683] domain-4: span=0-47 level=NUMA
[ 10.442687] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.442697] ERROR: groups don't span domain->span
[ 10.447452] domain-5: span=0-63 level=NUMA
[ 10.447456] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.447462] ERROR: groups don't span domain->span
[ 10.452223] CPU22 attaching sched-domain(s):
[ 10.452225] domain-0: span=20-23 level=MC
[ 10.452228] groups: 22:{ span=22 }, 23:{ span=23 }, 20:{ span=20 },
21:{ span=21 }
[ 10.452238] domain-1: span=16-31 level=DIE
[ 10.452241] groups: 20:{ span=20-23 cap=4096 }, 24:{ span=24-27
cap=4096 }, 28:{ span=28-31 cap=4096 }, 16:{ span=16-19 cap=4095 }
[ 10.452254] domain-2: span=16-31 level=NODE
[ 10.452257] groups: 16:{ span=16-31 cap=16383 }
[ 10.452262] domain-3: span=0-31 level=NUMA
[ 10.452265] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16382 }
[ 10.452273] domain-4: span=0-47 level=NUMA
[ 10.452276] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.452286] ERROR: groups don't span domain->span
[ 10.457042] domain-5: span=0-63 level=NUMA
[ 10.457046] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.457052] ERROR: groups don't span domain->span
[ 10.461813] CPU23 attaching sched-domain(s):
[ 10.461815] domain-0: span=20-23 level=MC
[ 10.461818] groups: 23:{ span=23 }, 20:{ span=20 }, 21:{ span=21 },
22:{ span=22 }
[ 10.461827] domain-1: span=16-31 level=DIE
[ 10.461831] groups: 20:{ span=20-23 cap=4096 }, 24:{ span=24-27
cap=4096 }, 28:{ span=28-31 cap=4096 }, 16:{ span=16-19 cap=4095 }
[ 10.461843] domain-2: span=16-31 level=NODE
[ 10.461846] groups: 16:{ span=16-31 cap=16383 }
[ 10.461852] domain-3: span=0-31 level=NUMA
[ 10.461855] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16382 }
[ 10.461862] domain-4: span=0-47 level=NUMA
[ 10.461866] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.461876] ERROR: groups don't span domain->span
[ 10.466632] domain-5: span=0-63 level=NUMA
[ 10.466635] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.466641] ERROR: groups don't span domain->span
[ 10.471403] CPU24 attaching sched-domain(s):
[ 10.471404] domain-0: span=24-27 level=MC
[ 10.471407] groups: 24:{ span=24 }, 25:{ span=25 }, 26:{ span=26 },
27:{ span=27 }
[ 10.471417] domain-1: span=16-31 level=DIE
[ 10.471420] groups: 24:{ span=24-27 cap=4096 }, 28:{ span=28-31
cap=4096 }, 16:{ span=16-19 cap=4095 }, 20:{ span=20-23 cap=4096 }
[ 10.471433] domain-2: span=16-31 level=NODE
[ 10.471436] groups: 16:{ span=16-31 cap=16383 }
[ 10.471441] domain-3: span=0-31 level=NUMA
[ 10.471445] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16382 }
[ 10.471452] domain-4: span=0-47 level=NUMA
[ 10.471456] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.471466] ERROR: groups don't span domain->span
[ 10.476221] domain-5: span=0-63 level=NUMA
[ 10.476225] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.476231] ERROR: groups don't span domain->span
[ 10.480992] CPU25 attaching sched-domain(s):
[ 10.480994] domain-0: span=24-27 level=MC
[ 10.480997] groups: 25:{ span=25 }, 26:{ span=26 }, 27:{ span=27 },
24:{ span=24 }
[ 10.481007] domain-1: span=16-31 level=DIE
[ 10.481010] groups: 24:{ span=24-27 cap=4096 }, 28:{ span=28-31
cap=4096 }, 16:{ span=16-19 cap=4095 }, 20:{ span=20-23 cap=4096 }
[ 10.481023] domain-2: span=16-31 level=NODE
[ 10.481026] groups: 16:{ span=16-31 cap=16383 }
[ 10.481031] domain-3: span=0-31 level=NUMA
[ 10.481034] groups: 16:{ span=16-31 cap=16384 }, 0:{ span=0-15
cap=16382 }
[ 10.481042] domain-4: span=0-47 level=NUMA
[ 10.481045] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.481055] ERROR: groups don't span domain->span
[ 10.485811] domain-5: span=0-63 level=NUMA
[ 10.485814] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.485820] ERROR: groups don't span domain->span
[ 10.490581] CPU26 attaching sched-domain(s):
[ 10.490583] domain-0: span=24-27 level=MC
[ 10.490586] groups: 26:{ span=26 }, 27:{ span=27 }, 24:{ span=24 },
25:{ span=25 }
[ 10.490596] domain-1: span=16-31 level=DIE
[ 10.490599] groups: 24:{ span=24-27 cap=4096 }, 28:{ span=28-31
cap=4096 }, 16:{ span=16-19 cap=4095 }, 20:{ span=20-23 cap=4096 }
[ 10.490611] domain-2: span=16-31 level=NODE
[ 10.490615] groups: 16:{ span=16-31 cap=16383 }
[ 10.490619] domain-3: span=0-31 level=NUMA
[ 10.490623] groups: 16:{ span=16-31 cap=16383 }, 0:{ span=0-15
cap=16382 }
[ 10.490630] domain-4: span=0-47 level=NUMA
[ 10.490634] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.490644] ERROR: groups don't span domain->span
[ 10.495400] domain-5: span=0-63 level=NUMA
[ 10.495403] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.495410] ERROR: groups don't span domain->span
[ 10.500170] CPU27 attaching sched-domain(s):
[ 10.500172] domain-0: span=24-27 level=MC
[ 10.500175] groups: 27:{ span=27 }, 24:{ span=24 }, 25:{ span=25 },
26:{ span=26 }
[ 10.500185] domain-1: span=16-31 level=DIE
[ 10.500188] groups: 24:{ span=24-27 cap=4096 }, 28:{ span=28-31
cap=4096 }, 16:{ span=16-19 cap=4095 }, 20:{ span=20-23 cap=4096 }
[ 10.500201] domain-2: span=16-31 level=NODE
[ 10.500204] groups: 16:{ span=16-31 cap=16383 }
[ 10.500209] domain-3: span=0-31 level=NUMA
[ 10.500212] groups: 16:{ span=16-31 cap=16383 }, 0:{ span=0-15
cap=16382 }
[ 10.500220] domain-4: span=0-47 level=NUMA
[ 10.500224] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.500234] ERROR: groups don't span domain->span
[ 10.504990] domain-5: span=0-63 level=NUMA
[ 10.504994] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.505000] ERROR: groups don't span domain->span
[ 10.509761] CPU28 attaching sched-domain(s):
[ 10.509763] domain-0: span=28-31 level=MC
[ 10.509766] groups: 28:{ span=28 }, 29:{ span=29 }, 30:{ span=30 },
31:{ span=31 }
[ 10.509775] domain-1: span=16-31 level=DIE
[ 10.509778] groups: 28:{ span=28-31 cap=4096 }, 16:{ span=16-19
cap=4095 }, 20:{ span=20-23 cap=4096 }, 24:{ span=24-27 cap=4096 }
[ 10.509791] domain-2: span=16-31 level=NODE
[ 10.509794] groups: 16:{ span=16-31 cap=16383 }
[ 10.509799] domain-3: span=0-31 level=NUMA
[ 10.509803] groups: 16:{ span=16-31 cap=16383 }, 0:{ span=0-15
cap=16382 }
[ 10.509810] domain-4: span=0-47 level=NUMA
[ 10.509814] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.509824] ERROR: groups don't span domain->span
[ 10.514580] domain-5: span=0-63 level=NUMA
[ 10.514583] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.514590] ERROR: groups don't span domain->span
[ 10.519350] CPU29 attaching sched-domain(s):
[ 10.519352] domain-0: span=28-31 level=MC
[ 10.519355] groups: 29:{ span=29 }, 30:{ span=30 }, 31:{ span=31 },
28:{ span=28 }
[ 10.519365] domain-1: span=16-31 level=DIE
[ 10.519368] groups: 28:{ span=28-31 cap=4096 }, 16:{ span=16-19
cap=4095 }, 20:{ span=20-23 cap=4096 }, 24:{ span=24-27 cap=4096 }
[ 10.519381] domain-2: span=16-31 level=NODE
[ 10.519384] groups: 16:{ span=16-31 cap=16383 }
[ 10.519388] domain-3: span=0-31 level=NUMA
[ 10.519392] groups: 16:{ span=16-31 cap=16383 }, 0:{ span=0-15
cap=16381 }
[ 10.519400] domain-4: span=0-47 level=NUMA
[ 10.519404] groups: 16:{ span=0-31 mask=16-31 cap=32766 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.519414] ERROR: groups don't span domain->span
[ 10.524169] domain-5: span=0-63 level=NUMA
[ 10.524173] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.524179] ERROR: groups don't span domain->span
[ 10.528940] CPU30 attaching sched-domain(s):
[ 10.528942] domain-0: span=28-31 level=MC
[ 10.528945] groups: 30:{ span=30 }, 31:{ span=31 }, 28:{ span=28 },
29:{ span=29 }
[ 10.528955] domain-1: span=16-31 level=DIE
[ 10.528958] groups: 28:{ span=28-31 cap=4096 }, 16:{ span=16-19
cap=4095 }, 20:{ span=20-23 cap=4096 }, 24:{ span=24-27 cap=4096 }
[ 10.528971] domain-2: span=16-31 level=NODE
[ 10.528974] groups: 16:{ span=16-31 cap=16383 }
[ 10.528979] domain-3: span=0-31 level=NUMA
[ 10.528982] groups: 16:{ span=16-31 cap=16383 }, 0:{ span=0-15
cap=16381 }
[ 10.528990] domain-4: span=0-47 level=NUMA
[ 10.528993] groups: 16:{ span=0-31 mask=16-31 cap=32762 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.529004] ERROR: groups don't span domain->span
[ 10.533759] domain-5: span=0-63 level=NUMA
[ 10.533763] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.533769] ERROR: groups don't span domain->span
[ 10.538529] CPU31 attaching sched-domain(s):
[ 10.538531] domain-0: span=28-31 level=MC
[ 10.538534] groups: 31:{ span=31 }, 28:{ span=28 }, 29:{ span=29 },
30:{ span=30 }
[ 10.538544] domain-1: span=16-31 level=DIE
[ 10.538547] groups: 28:{ span=28-31 cap=4096 }, 16:{ span=16-19
cap=4095 }, 20:{ span=20-23 cap=4096 }, 24:{ span=24-27 cap=4096 }
[ 10.538560] domain-2: span=16-31 level=NODE
[ 10.538563] groups: 16:{ span=16-31 cap=16383 }
[ 10.538567] domain-3: span=0-31 level=NUMA
[ 10.538571] groups: 16:{ span=16-31 cap=16383 }, 0:{ span=0-15
cap=16381 }
[ 10.538579] domain-4: span=0-47 level=NUMA
[ 10.538582] groups: 16:{ span=0-31 mask=16-31 cap=32762 }, 32:{
span=0-15,32-63 mask=32-47 cap=49152 }
[ 10.538592] ERROR: groups don't span domain->span
[ 10.543347] domain-5: span=0-63 level=NUMA
[ 10.543351] groups: 16:{ span=0-47 mask=16-31 cap=49150 }
[ 10.543357] ERROR: groups don't span domain->span
[ 10.548115] CPU32 attaching sched-domain(s):
[ 10.548117] domain-0: span=32-35 level=MC
[ 10.548119] groups: 32:{ span=32 }, 33:{ span=33 }, 34:{ span=34 },
35:{ span=35 }
[ 10.548129] domain-1: span=32-47 level=DIE
[ 10.548132] groups: 32:{ span=32-35 cap=4096 }, 36:{ span=36-39
cap=4096 }, 40:{ span=40-43 cap=4096 }, 44:{ span=44-47 cap=4096 }
[ 10.548145] domain-2: span=32-47 level=NODE
[ 10.548147] groups: 32:{ span=32-47 cap=16384 }
[ 10.548152] domain-3: span=32-63 level=NUMA
[ 10.548156] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.548165] ERROR: groups don't span domain->span
[ 10.552921] ERROR: parent span is not a superset of domain->span
[ 10.558994] domain-4: span=0-15,32-63 level=NUMA
[ 10.558998] groups: 32:{ span=32-63 mask=32-47 cap=32768 }, 0:{
span=0-31 cap=32763 }
[ 10.559007] ERROR: groups don't span domain->span
[ 10.563762] domain-5: span=0-63 level=NUMA
[ 10.563766] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49152
}, 16:{ span=0-31 mask=16-31 cap=32762 }
[ 10.563780] CPU33 attaching sched-domain(s):
[ 10.563782] domain-0: span=32-35 level=MC
[ 10.563785] groups: 33:{ span=33 }, 34:{ span=34 }, 35:{ span=35 },
32:{ span=32 }
[ 10.563794] domain-1: span=32-47 level=DIE
[ 10.563798] groups: 32:{ span=32-35 cap=4096 }, 36:{ span=36-39
cap=4096 }, 40:{ span=40-43 cap=4096 }, 44:{ span=44-47 cap=4096 }
[ 10.563810] domain-2: span=32-47 level=NODE
[ 10.563813] groups: 32:{ span=32-47 cap=16384 }
[ 10.563818] domain-3: span=32-63 level=NUMA
[ 10.563821] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.563830] ERROR: groups don't span domain->span
[ 10.568586] ERROR: parent span is not a superset of domain->span
[ 10.574660] domain-4: span=0-15,32-63 level=NUMA
[ 10.574664] groups: 32:{ span=32-63 mask=32-47 cap=32768 }, 0:{
span=0-31 cap=32763 }
[ 10.574673] ERROR: groups don't span domain->span
[ 10.579428] domain-5: span=0-63 level=NUMA
[ 10.579431] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32762 }
[ 10.579446] CPU34 attaching sched-domain(s):
[ 10.579448] domain-0: span=32-35 level=MC
[ 10.579450] groups: 34:{ span=34 }, 35:{ span=35 }, 32:{ span=32 },
33:{ span=33 }
[ 10.579460] domain-1: span=32-47 level=DIE
[ 10.579464] groups: 32:{ span=32-35 cap=4096 }, 36:{ span=36-39
cap=4096 }, 40:{ span=40-43 cap=4096 }, 44:{ span=44-47 cap=4096 }
[ 10.579477] domain-2: span=32-47 level=NODE
[ 10.579480] groups: 32:{ span=32-47 cap=16384 }
[ 10.579485] domain-3: span=32-63 level=NUMA
[ 10.579488] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.579498] ERROR: groups don't span domain->span
[ 10.584253] ERROR: parent span is not a superset of domain->span
[ 10.590327] domain-4: span=0-15,32-63 level=NUMA
[ 10.590330] groups: 32:{ span=32-63 mask=32-47 cap=32768 }, 0:{
span=0-31 cap=32763 }
[ 10.590339] ERROR: groups don't span domain->span
[ 10.595094] domain-5: span=0-63 level=NUMA
[ 10.595097] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32762 }
[ 10.595111] CPU35 attaching sched-domain(s):
[ 10.595113] domain-0: span=32-35 level=MC
[ 10.595116] groups: 35:{ span=35 }, 32:{ span=32 cap=1022 }, 33:{
span=33 }, 34:{ span=34 }
[ 10.595127] domain-1: span=32-47 level=DIE
[ 10.595130] groups: 32:{ span=32-35 cap=4096 }, 36:{ span=36-39
cap=4096 }, 40:{ span=40-43 cap=4096 }, 44:{ span=44-47 cap=4096 }
[ 10.595143] domain-2: span=32-47 level=NODE
[ 10.595146] groups: 32:{ span=32-47 cap=16384 }
[ 10.595151] domain-3: span=32-63 level=NUMA
[ 10.595154] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.595163] ERROR: groups don't span domain->span
[ 10.599918] ERROR: parent span is not a superset of domain->span
[ 10.605992] domain-4: span=0-15,32-63 level=NUMA
[ 10.605995] groups: 32:{ span=32-63 mask=32-47 cap=32768 }, 0:{
span=0-31 cap=32763 }
[ 10.606004] ERROR: groups don't span domain->span
[ 10.610759] domain-5: span=0-63 level=NUMA
[ 10.610763] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32762 }
[ 10.610777] CPU36 attaching sched-domain(s):
[ 10.610779] domain-0: span=36-39 level=MC
[ 10.610782] groups: 36:{ span=36 }, 37:{ span=37 }, 38:{ span=38 },
39:{ span=39 }
[ 10.610792] domain-1: span=32-47 level=DIE
[ 10.610795] groups: 36:{ span=36-39 cap=4096 }, 40:{ span=40-43
cap=4096 }, 44:{ span=44-47 cap=4096 }, 32:{ span=32-35 cap=4096 }
[ 10.610807] domain-2: span=32-47 level=NODE
[ 10.610810] groups: 32:{ span=32-47 cap=16384 }
[ 10.610815] domain-3: span=32-63 level=NUMA
[ 10.610818] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.610827] ERROR: groups don't span domain->span
[ 10.615582] ERROR: parent span is not a superset of domain->span
[ 10.621656] domain-4: span=0-15,32-63 level=NUMA
[ 10.621660] groups: 32:{ span=32-63 mask=32-47 cap=32768 }, 0:{
span=0-31 cap=32763 }
[ 10.621668] ERROR: groups don't span domain->span
[ 10.626424] domain-5: span=0-63 level=NUMA
[ 10.626427] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32762 }
[ 10.626442] CPU37 attaching sched-domain(s):
[ 10.626444] domain-0: span=36-39 level=MC
[ 10.626447] groups: 37:{ span=37 }, 38:{ span=38 }, 39:{ span=39 },
36:{ span=36 }
[ 10.626456] domain-1: span=32-47 level=DIE
[ 10.626459] groups: 36:{ span=36-39 cap=4096 }, 40:{ span=40-43
cap=4096 }, 44:{ span=44-47 cap=4096 }, 32:{ span=32-35 cap=4095 }
[ 10.626472] domain-2: span=32-47 level=NODE
[ 10.626475] groups: 32:{ span=32-47 cap=16383 }
[ 10.626480] domain-3: span=32-63 level=NUMA
[ 10.626483] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.626492] ERROR: groups don't span domain->span
[ 10.631248] ERROR: parent span is not a superset of domain->span
[ 10.637322] domain-4: span=0-15,32-63 level=NUMA
[ 10.637325] groups: 32:{ span=32-63 mask=32-47 cap=32768 }, 0:{
span=0-31 cap=32763 }
[ 10.637334] ERROR: groups don't span domain->span
[ 10.642089] domain-5: span=0-63 level=NUMA
[ 10.642093] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32762 }
[ 10.642108] CPU38 attaching sched-domain(s):
[ 10.642109] domain-0: span=36-39 level=MC
[ 10.642112] groups: 38:{ span=38 }, 39:{ span=39 }, 36:{ span=36
cap=1023 }, 37:{ span=37 }
[ 10.642123] domain-1: span=32-47 level=DIE
[ 10.642126] groups: 36:{ span=36-39 cap=4096 }, 40:{ span=40-43
cap=4096 }, 44:{ span=44-47 cap=4096 }, 32:{ span=32-35 cap=4095 }
[ 10.642138] domain-2: span=32-47 level=NODE
[ 10.642141] groups: 32:{ span=32-47 cap=16383 }
[ 10.642146] domain-3: span=32-63 level=NUMA
[ 10.642149] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.642158] ERROR: groups don't span domain->span
[ 10.646914] ERROR: parent span is not a superset of domain->span
[ 10.652988] domain-4: span=0-15,32-63 level=NUMA
[ 10.652991] groups: 32:{ span=32-63 mask=32-47 cap=32766 }, 0:{
span=0-31 cap=32763 }
[ 10.653001] ERROR: groups don't span domain->span
[ 10.657756] domain-5: span=0-63 level=NUMA
[ 10.657759] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.657773] CPU39 attaching sched-domain(s):
[ 10.657776] domain-0: span=36-39 level=MC
[ 10.657778] groups: 39:{ span=39 }, 36:{ span=36 cap=1023 }, 37:{
span=37 }, 38:{ span=38 }
[ 10.657789] domain-1: span=32-47 level=DIE
[ 10.657792] groups: 36:{ span=36-39 cap=4096 }, 40:{ span=40-43
cap=4096 }, 44:{ span=44-47 cap=4096 }, 32:{ span=32-35 cap=4095 }
[ 10.657805] domain-2: span=32-47 level=NODE
[ 10.657808] groups: 32:{ span=32-47 cap=16383 }
[ 10.657813] domain-3: span=32-63 level=NUMA
[ 10.657816] groups: 32:{ span=32-47 cap=16384 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.657824] ERROR: groups don't span domain->span
[ 10.662580] ERROR: parent span is not a superset of domain->span
[ 10.668655] domain-4: span=0-15,32-63 level=NUMA
[ 10.668658] groups: 32:{ span=32-63 mask=32-47 cap=32766 }, 0:{
span=0-31 cap=32762 }
[ 10.668667] ERROR: groups don't span domain->span
[ 10.673422] domain-5: span=0-63 level=NUMA
[ 10.673426] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.673440] CPU40 attaching sched-domain(s):
[ 10.673441] domain-0: span=40-43 level=MC
[ 10.673444] groups: 40:{ span=40 }, 41:{ span=41 }, 42:{ span=42 },
43:{ span=43 }
[ 10.673454] domain-1: span=32-47 level=DIE
[ 10.673457] groups: 40:{ span=40-43 cap=4096 }, 44:{ span=44-47
cap=4096 }, 32:{ span=32-35 cap=4095 }, 36:{ span=36-39 cap=4096 }
[ 10.673469] domain-2: span=32-47 level=NODE
[ 10.673472] groups: 32:{ span=32-47 cap=16383 }
[ 10.673477] domain-3: span=32-63 level=NUMA
[ 10.673480] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.673490] ERROR: groups don't span domain->span
[ 10.678245] ERROR: parent span is not a superset of domain->span
[ 10.684319] domain-4: span=0-15,32-63 level=NUMA
[ 10.684323] groups: 32:{ span=32-63 mask=32-47 cap=32766 }, 0:{
span=0-31 cap=32762 }
[ 10.684331] ERROR: groups don't span domain->span
[ 10.689087] domain-5: span=0-63 level=NUMA
[ 10.689090] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.689104] CPU41 attaching sched-domain(s):
[ 10.689105] domain-0: span=40-43 level=MC
[ 10.689108] groups: 41:{ span=41 }, 42:{ span=42 }, 43:{ span=43 },
40:{ span=40 }
[ 10.689118] domain-1: span=32-47 level=DIE
[ 10.689121] groups: 40:{ span=40-43 cap=4096 }, 44:{ span=44-47
cap=4096 }, 32:{ span=32-35 cap=4095 }, 36:{ span=36-39 cap=4096 }
[ 10.689133] domain-2: span=32-47 level=NODE
[ 10.689136] groups: 32:{ span=32-47 cap=16383 }
[ 10.689141] domain-3: span=32-63 level=NUMA
[ 10.689145] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.689154] ERROR: groups don't span domain->span
[ 10.693909] ERROR: parent span is not a superset of domain->span
[ 10.699982] domain-4: span=0-15,32-63 level=NUMA
[ 10.699985] groups: 32:{ span=32-63 mask=32-47 cap=32766 }, 0:{
span=0-31 cap=32762 }
[ 10.699994] ERROR: groups don't span domain->span
[ 10.704749] domain-5: span=0-63 level=NUMA
[ 10.704753] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.704767] CPU42 attaching sched-domain(s):
[ 10.704769] domain-0: span=40-43 level=MC
[ 10.704771] groups: 42:{ span=42 }, 43:{ span=43 }, 40:{ span=40 },
41:{ span=41 }
[ 10.704781] domain-1: span=32-47 level=DIE
[ 10.704784] groups: 40:{ span=40-43 cap=4096 }, 44:{ span=44-47
cap=4096 }, 32:{ span=32-35 cap=4095 }, 36:{ span=36-39 cap=4096 }
[ 10.704797] domain-2: span=32-47 level=NODE
[ 10.704801] groups: 32:{ span=32-47 cap=16383 }
[ 10.704806] domain-3: span=32-63 level=NUMA
[ 10.704809] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.704818] ERROR: groups don't span domain->span
[ 10.709573] ERROR: parent span is not a superset of domain->span
[ 10.715651] domain-4: span=0-15,32-63 level=NUMA
[ 10.715655] groups: 32:{ span=32-63 mask=32-47 cap=32766 }, 0:{
span=0-31 cap=32762 }
[ 10.715664] ERROR: groups don't span domain->span
[ 10.720420] domain-5: span=0-63 level=NUMA
[ 10.720424] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.720438] CPU43 attaching sched-domain(s):
[ 10.720440] domain-0: span=40-43 level=MC
[ 10.720443] groups: 43:{ span=43 }, 40:{ span=40 }, 41:{ span=41 },
42:{ span=42 }
[ 10.720453] domain-1: span=32-47 level=DIE
[ 10.720456] groups: 40:{ span=40-43 cap=4096 }, 44:{ span=44-47
cap=4096 }, 32:{ span=32-35 cap=4095 }, 36:{ span=36-39 cap=4096 }
[ 10.720469] domain-2: span=32-47 level=NODE
[ 10.720472] groups: 32:{ span=32-47 cap=16383 }
[ 10.720477] domain-3: span=32-63 level=NUMA
[ 10.720481] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.720490] ERROR: groups don't span domain->span
[ 10.725246] ERROR: parent span is not a superset of domain->span
[ 10.731319] domain-4: span=0-15,32-63 level=NUMA
[ 10.731323] groups: 32:{ span=32-63 mask=32-47 cap=32767 }, 0:{
span=0-31 cap=32762 }
[ 10.731332] ERROR: groups don't span domain->span
[ 10.736086] domain-5: span=0-63 level=NUMA
[ 10.736089] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.736104] CPU44 attaching sched-domain(s):
[ 10.736105] domain-0: span=44-47 level=MC
[ 10.736108] groups: 44:{ span=44 }, 45:{ span=45 }, 46:{ span=46 },
47:{ span=47 }
[ 10.736118] domain-1: span=32-47 level=DIE
[ 10.736121] groups: 44:{ span=44-47 cap=4096 }, 32:{ span=32-35
cap=4095 }, 36:{ span=36-39 cap=4096 }, 40:{ span=40-43 cap=4096 }
[ 10.736134] domain-2: span=32-47 level=NODE
[ 10.736137] groups: 32:{ span=32-47 cap=16383 }
[ 10.736142] domain-3: span=32-63 level=NUMA
[ 10.736145] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.736154] ERROR: groups don't span domain->span
[ 10.740909] ERROR: parent span is not a superset of domain->span
[ 10.746983] domain-4: span=0-15,32-63 level=NUMA
[ 10.746987] groups: 32:{ span=32-63 mask=32-47 cap=32767 }, 0:{
span=0-31 cap=32762 }
[ 10.746996] ERROR: groups don't span domain->span
[ 10.751751] domain-5: span=0-63 level=NUMA
[ 10.751754] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49149
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.751769] CPU45 attaching sched-domain(s):
[ 10.751771] domain-0: span=44-47 level=MC
[ 10.751773] groups: 45:{ span=45 }, 46:{ span=46 }, 47:{ span=47 },
44:{ span=44 }
[ 10.751783] domain-1: span=32-47 level=DIE
[ 10.751786] groups: 44:{ span=44-47 cap=4096 }, 32:{ span=32-35
cap=4095 }, 36:{ span=36-39 cap=4096 }, 40:{ span=40-43 cap=4096 }
[ 10.751799] domain-2: span=32-47 level=NODE
[ 10.751802] groups: 32:{ span=32-47 cap=16383 }
[ 10.751806] domain-3: span=32-63 level=NUMA
[ 10.751809] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.751818] ERROR: groups don't span domain->span
[ 10.756574] ERROR: parent span is not a superset of domain->span
[ 10.762648] domain-4: span=0-15,32-63 level=NUMA
[ 10.762651] groups: 32:{ span=32-63 mask=32-47 cap=32767 }, 0:{
span=0-31 cap=32762 }
[ 10.762660] ERROR: groups don't span domain->span
[ 10.767415] domain-5: span=0-63 level=NUMA
[ 10.767419] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49144
}, 16:{ span=0-31 mask=16-31 cap=32763 }
[ 10.767433] CPU46 attaching sched-domain(s):
[ 10.767435] domain-0: span=44-47 level=MC
[ 10.767438] groups: 46:{ span=46 }, 47:{ span=47 }, 44:{ span=44 },
45:{ span=45 }
[ 10.767448] domain-1: span=32-47 level=DIE
[ 10.767451] groups: 44:{ span=44-47 cap=4096 }, 32:{ span=32-35
cap=4095 }, 36:{ span=36-39 cap=4096 }, 40:{ span=40-43 cap=4096 }
[ 10.767464] domain-2: span=32-47 level=NODE
[ 10.767466] groups: 32:{ span=32-47 cap=16383 }
[ 10.767471] domain-3: span=32-63 level=NUMA
[ 10.767474] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.767484] ERROR: groups don't span domain->span
[ 10.772239] ERROR: parent span is not a superset of domain->span
[ 10.778312] domain-4: span=0-15,32-63 level=NUMA
[ 10.778316] groups: 32:{ span=32-63 mask=32-47 cap=32767 }, 0:{
span=0-31 cap=32762 }
[ 10.778325] ERROR: groups don't span domain->span
[ 10.783080] domain-5: span=0-63 level=NUMA
[ 10.783083] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49144
}, 16:{ span=0-31 mask=16-31 cap=32758 }
[ 10.783097] CPU47 attaching sched-domain(s):
[ 10.783099] domain-0: span=44-47 level=MC
[ 10.783102] groups: 47:{ span=47 }, 44:{ span=44 }, 45:{ span=45 },
46:{ span=46 }
[ 10.783112] domain-1: span=32-47 level=DIE
[ 10.783115] groups: 44:{ span=44-47 cap=4096 }, 32:{ span=32-35
cap=4095 }, 36:{ span=36-39 cap=4096 }, 40:{ span=40-43 cap=4096 }
[ 10.783128] domain-2: span=32-47 level=NODE
[ 10.783131] groups: 32:{ span=32-47 cap=16383 }
[ 10.783136] domain-3: span=32-63 level=NUMA
[ 10.783139] groups: 32:{ span=32-47 cap=16383 }, 48:{
span=0-31,48-63 mask=48-63 cap=49152 }
[ 10.783148] ERROR: groups don't span domain->span
[ 10.787904] ERROR: parent span is not a superset of domain->span
[ 10.793978] domain-4: span=0-15,32-63 level=NUMA
[ 10.793982] groups: 32:{ span=32-63 mask=32-47 cap=32767 }, 0:{
span=0-31 cap=32763 }
[ 10.793991] ERROR: groups don't span domain->span
[ 10.798746] domain-5: span=0-63 level=NUMA
[ 10.798749] groups: 32:{ span=0-15,32-63 mask=32-47 cap=49144
}, 16:{ span=0-31 mask=16-31 cap=32758 }
[ 10.798764] CPU48 attaching sched-domain(s):
[ 10.798766] domain-0: span=48-51 level=MC
[ 10.798769] groups: 48:{ span=48 }, 49:{ span=49 }, 50:{ span=50 },
51:{ span=51 }
[ 10.798779] domain-1: span=48-63 level=DIE
[ 10.798782] groups: 48:{ span=48-51 cap=4096 }, 52:{ span=52-55
cap=4096 }, 56:{ span=56-59 cap=4096 }, 60:{ span=60-63 cap=4096 }
[ 10.798795] domain-2: span=0-31,48-63 level=NODE
[ 10.798798] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.798809] domain-3: span=0-63 level=NUMA
[ 10.798812] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.798826] CPU49 attaching sched-domain(s):
[ 10.798828] domain-0: span=48-51 level=MC
[ 10.798830] groups: 49:{ span=49 }, 50:{ span=50 }, 51:{ span=51 },
48:{ span=48 }
[ 10.798840] domain-1: span=48-63 level=DIE
[ 10.798843] groups: 48:{ span=48-51 cap=4096 }, 52:{ span=52-55
cap=4096 }, 56:{ span=56-59 cap=4096 }, 60:{ span=60-63 cap=4096 }
[ 10.798856] domain-2: span=0-31,48-63 level=NODE
[ 10.798859] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.798870] domain-3: span=0-63 level=NUMA
[ 10.798873] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.798886] CPU50 attaching sched-domain(s):
[ 10.798887] domain-0: span=48-51 level=MC
[ 10.798890] groups: 50:{ span=50 }, 51:{ span=51 }, 48:{ span=48 },
49:{ span=49 }
[ 10.798900] domain-1: span=48-63 level=DIE
[ 10.798903] groups: 48:{ span=48-51 cap=4096 }, 52:{ span=52-55
cap=4096 }, 56:{ span=56-59 cap=4096 }, 60:{ span=60-63 cap=4096 }
[ 10.798916] domain-2: span=0-31,48-63 level=NODE
[ 10.798919] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.798930] domain-3: span=0-63 level=NUMA
[ 10.798933] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.798946] CPU51 attaching sched-domain(s):
[ 10.798948] domain-0: span=48-51 level=MC
[ 10.798950] groups: 51:{ span=51 }, 48:{ span=48 }, 49:{ span=49 },
50:{ span=50 }
[ 10.798960] domain-1: span=48-63 level=DIE
[ 10.798963] groups: 48:{ span=48-51 cap=4096 }, 52:{ span=52-55
cap=4096 }, 56:{ span=56-59 cap=4096 }, 60:{ span=60-63 cap=4096 }
[ 10.798976] domain-2: span=0-31,48-63 level=NODE
[ 10.798979] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.798990] domain-3: span=0-63 level=NUMA
[ 10.798993] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799006] CPU52 attaching sched-domain(s):
[ 10.799008] domain-0: span=52-55 level=MC
[ 10.799011] groups: 52:{ span=52 }, 53:{ span=53 }, 54:{ span=54 },
55:{ span=55 }
[ 10.799021] domain-1: span=48-63 level=DIE
[ 10.799023] groups: 52:{ span=52-55 cap=4096 }, 56:{ span=56-59
cap=4096 }, 60:{ span=60-63 cap=4096 }, 48:{ span=48-51 cap=4096 }
[ 10.799036] domain-2: span=0-31,48-63 level=NODE
[ 10.799039] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799050] domain-3: span=0-63 level=NUMA
[ 10.799054] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799066] CPU53 attaching sched-domain(s):
[ 10.799068] domain-0: span=52-55 level=MC
[ 10.799071] groups: 53:{ span=53 }, 54:{ span=54 }, 55:{ span=55 },
52:{ span=52 }
[ 10.799081] domain-1: span=48-63 level=DIE
[ 10.799084] groups: 52:{ span=52-55 cap=4096 }, 56:{ span=56-59
cap=4096 }, 60:{ span=60-63 cap=4096 }, 48:{ span=48-51 cap=4096 }
[ 10.799097] domain-2: span=0-31,48-63 level=NODE
[ 10.799100] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799110] domain-3: span=0-63 level=NUMA
[ 10.799114] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799127] CPU54 attaching sched-domain(s):
[ 10.799129] domain-0: span=52-55 level=MC
[ 10.799131] groups: 54:{ span=54 }, 55:{ span=55 }, 52:{ span=52 },
53:{ span=53 }
[ 10.799141] domain-1: span=48-63 level=DIE
[ 10.799144] groups: 52:{ span=52-55 cap=4096 }, 56:{ span=56-59
cap=4096 }, 60:{ span=60-63 cap=4096 }, 48:{ span=48-51 cap=4096 }
[ 10.799157] domain-2: span=0-31,48-63 level=NODE
[ 10.799161] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799171] domain-3: span=0-63 level=NUMA
[ 10.799175] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799188] CPU55 attaching sched-domain(s):
[ 10.799189] domain-0: span=52-55 level=MC
[ 10.799192] groups: 55:{ span=55 }, 52:{ span=52 }, 53:{ span=53 },
54:{ span=54 }
[ 10.799202] domain-1: span=48-63 level=DIE
[ 10.799205] groups: 52:{ span=52-55 cap=4096 }, 56:{ span=56-59
cap=4096 }, 60:{ span=60-63 cap=4096 }, 48:{ span=48-51 cap=4096 }
[ 10.799218] domain-2: span=0-31,48-63 level=NODE
[ 10.799221] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799231] domain-3: span=0-63 level=NUMA
[ 10.799235] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799248] CPU56 attaching sched-domain(s):
[ 10.799249] domain-0: span=56-59 level=MC
[ 10.799252] groups: 56:{ span=56 }, 57:{ span=57 }, 58:{ span=58 },
59:{ span=59 }
[ 10.799262] domain-1: span=48-63 level=DIE
[ 10.799265] groups: 56:{ span=56-59 cap=4096 }, 60:{ span=60-63
cap=4096 }, 48:{ span=48-51 cap=4096 }, 52:{ span=52-55 cap=4096 }
[ 10.799278] domain-2: span=0-31,48-63 level=NODE
[ 10.799281] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799292] domain-3: span=0-63 level=NUMA
[ 10.799295] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799308] CPU57 attaching sched-domain(s):
[ 10.799310] domain-0: span=56-59 level=MC
[ 10.799313] groups: 57:{ span=57 }, 58:{ span=58 }, 59:{ span=59 },
56:{ span=56 }
[ 10.799322] domain-1: span=48-63 level=DIE
[ 10.799325] groups: 56:{ span=56-59 cap=4096 }, 60:{ span=60-63
cap=4096 }, 48:{ span=48-51 cap=4096 }, 52:{ span=52-55 cap=4096 }
[ 10.799338] domain-2: span=0-31,48-63 level=NODE
[ 10.799341] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799352] domain-3: span=0-63 level=NUMA
[ 10.799355] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799368] CPU58 attaching sched-domain(s):
[ 10.799370] domain-0: span=56-59 level=MC
[ 10.799373] groups: 58:{ span=58 }, 59:{ span=59 }, 56:{ span=56 },
57:{ span=57 }
[ 10.799382] domain-1: span=48-63 level=DIE
[ 10.799385] groups: 56:{ span=56-59 cap=4096 }, 60:{ span=60-63
cap=4096 }, 48:{ span=48-51 cap=4096 }, 52:{ span=52-55 cap=4096 }
[ 10.799397] domain-2: span=0-31,48-63 level=NODE
[ 10.799400] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799410] domain-3: span=0-63 level=NUMA
[ 10.799414] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799426] CPU59 attaching sched-domain(s):
[ 10.799428] domain-0: span=56-59 level=MC
[ 10.799431] groups: 59:{ span=59 }, 56:{ span=56 }, 57:{ span=57 },
58:{ span=58 }
[ 10.799440] domain-1: span=48-63 level=DIE
[ 10.799443] groups: 56:{ span=56-59 cap=4096 }, 60:{ span=60-63
cap=4096 }, 48:{ span=48-51 cap=4096 }, 52:{ span=52-55 cap=4096 }
[ 10.799456] domain-2: span=0-31,48-63 level=NODE
[ 10.799459] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799469] domain-3: span=0-63 level=NUMA
[ 10.799473] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799485] CPU60 attaching sched-domain(s):
[ 10.799487] domain-0: span=60-63 level=MC
[ 10.799490] groups: 60:{ span=60 }, 61:{ span=61 }, 62:{ span=62 },
63:{ span=63 }
[ 10.799500] domain-1: span=48-63 level=DIE
[ 10.799503] groups: 60:{ span=60-63 cap=4096 }, 48:{ span=48-51
cap=4096 }, 52:{ span=52-55 cap=4096 }, 56:{ span=56-59 cap=4096 }
[ 10.799515] domain-2: span=0-31,48-63 level=NODE
[ 10.799519] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799529] domain-3: span=0-63 level=NUMA
[ 10.799533] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799546] CPU61 attaching sched-domain(s):
[ 10.799548] domain-0: span=60-63 level=MC
[ 10.799550] groups: 61:{ span=61 }, 62:{ span=62 }, 63:{ span=63 },
60:{ span=60 }
[ 10.799560] domain-1: span=48-63 level=DIE
[ 10.799563] groups: 60:{ span=60-63 cap=4096 }, 48:{ span=48-51
cap=4096 }, 52:{ span=52-55 cap=4096 }, 56:{ span=56-59 cap=4096 }
[ 10.799575] domain-2: span=0-31,48-63 level=NODE
[ 10.799579] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799589] domain-3: span=0-63 level=NUMA
[ 10.799593] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799606] CPU62 attaching sched-domain(s):
[ 10.799607] domain-0: span=60-63 level=MC
[ 10.799610] groups: 62:{ span=62 }, 63:{ span=63 }, 60:{ span=60 },
61:{ span=61 }
[ 10.799620] domain-1: span=48-63 level=DIE
[ 10.799623] groups: 60:{ span=60-63 cap=4096 }, 48:{ span=48-51
cap=4096 }, 52:{ span=52-55 cap=4096 }, 56:{ span=56-59 cap=4096 }
[ 10.799635] domain-2: span=0-31,48-63 level=NODE
[ 10.799639] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799649] domain-3: span=0-63 level=NUMA
[ 10.799653] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799665] CPU63 attaching sched-domain(s):
[ 10.799667] domain-0: span=60-63 level=MC
[ 10.799670] groups: 63:{ span=63 }, 60:{ span=60 }, 61:{ span=61 },
62:{ span=62 }
[ 10.799680] domain-1: span=48-63 level=DIE
[ 10.799683] groups: 60:{ span=60-63 cap=4096 }, 48:{ span=48-51
cap=4096 }, 52:{ span=52-55 cap=4096 }, 56:{ span=56-59 cap=4096 }
[ 10.799695] domain-2: span=0-31,48-63 level=NODE
[ 10.799699] groups: 48:{ span=48-63 cap=16384 }, 0:{ span=0-15
cap=16378 }, 16:{ span=16-31 cap=16383 }
[ 10.799709] domain-3: span=0-63 level=NUMA
[ 10.799713] groups: 48:{ span=0-31,48-63 mask=48-63 cap=49152 },
32:{ span=32-47 cap=16383 }
[ 10.799726] root domain span: 0-63 (max cpu_capacity = 1024)
[ 10.833135] devtmpfs: initialized
[ 10.850493] clocksource: jiffies: mask: 0xffffffff max_cycles:
0xffffffff, max_idle_ns: 7645041785100000 ns
[ 10.860707] futex hash table entries: 16384 (order: 8, 1048576 bytes)
[ 10.870358] pinctrl core: initialized pinctrl subsystem
[ 10.877601] SMBIOS 3.0.0 present.
[ 10.880985] DMI: Huawei Taishan 2280 /D05, BIOS Hisilicon D05 IT17
Nemo 2.0 RC0 10/05/2018
[ 10.892569] cpuidle: using governor menu
[ 10.897069] vdso: 2 pages (1 code @ (____ptrval____), 1 data @
(____ptrval____))
[ 10.904605] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 10.913915] DMA: preallocated 256 KiB pool for atomic allocations
[ 10.921368] Serial: AMBA PL011 UART driver
[ 10.952984] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[ 10.960757] cryptd: max_cpu_qlen set to 1000
[ 10.966351] ACPI: Interpreter disabled.
[ 10.970932] vgaarb: loaded
[ 10.974738] SCSI subsystem initialized
[ 10.978927] libata version 3.00 loaded.
[ 10.979611] Registered efivars operations
[ 10.985647] clocksource: Switched to clocksource arch_sys_counter
[ 10.992239] VFS: Disk quotas dquot_6.6.0
[ 10.996315] VFS: Dquot-cache hash table entries: 512 (order 0, 4096
bytes)
[ 11.003588] pnp: PnP ACPI: disabled
[ 11.016949] PCI: CLS 0 bytes, default 64
[ 11.017166] Unpacking initramfs...
[ 18.701705] Freeing initrd memory: 258820K
[ 18.712591] hw perfevents: enabled with armv8_cortex_a72 PMU driver,
7 counters available
[ 18.721384] kvm [1]: 8-bit VMID
[ 18.724582] kvm [1]: IPA Size Limit: 44bits
[ 18.730467] kvm [1]: GICv4 support disabled
[ 18.734731] kvm [1]: vgic-v2@fe020000
[ 18.738493] kvm [1]: GIC system register CPU interface enabled
[ 18.745755] kvm [1]: vgic interrupt IRQ1
[ 18.751000] kvm [1]: Hyp mode initialized successfully
[ 18.833687] workingset: timestamp_bits=44 max_order=25 bucket_order=0
[ 18.852508] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[ 18.890310] Block layer SCSI generic (bsg) driver version 0.4 loaded
(major 250)
[ 18.897849] io scheduler noop registered
[ 18.901841] io scheduler deadline registered
[ 18.906503] io scheduler cfq registered (default)
[ 18.911292] io scheduler mq-deadline registered
[ 18.915903] io scheduler kyber registered
[ 18.927845] hisi-lpc a01b0000.isa: registered range
[0x0000000000ffbfff - 0x0000000000ffffff]
[ 18.943828] hisi-pcie-almost-ecam af800000.pcie: host bridge
/soc/pcie@a00a0000 ranges:
[ 18.952004] hisi-pcie-almost-ecam af800000.pcie: MEM
0xa8000000..0xaf7effff -> 0xa8000000
[ 18.960522] hisi-pcie-almost-ecam af800000.pcie: IO
0xaf7f0000..0xaf7fffff -> 0x00000000
[ 18.969053] hisi-pcie-almost-ecam af800000.pcie: resource collision:
[io 0x0000-0xffff] conflicts with IPMI Address 1 [io 0x00e4]
[ 18.981149] hisi-pcie-almost-ecam: probe of af800000.pcie failed with
error -16
[ 18.989415] EINJ: ACPI disabled.
[ 19.003110] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 19.012350] SuperH (H)SCI(F) driver initialized
[ 19.017498] msm_serial: driver initialized
[ 19.022268] arm-smmu-v3 d0040000.smmu_alg: option
hisilicon,broken-prefetch-cmd
[ 19.030146] arm-smmu-v3 d0040000.smmu_alg: ias 44-bit, oas 44-bit
(features 0x00000f0d)
[ 19.042717] arm-smmu-v3 8d0040000.smmu_alg: option
hisilicon,broken-prefetch-cmd
[ 19.050616] arm-smmu-v3 8d0040000.smmu_alg: ias 44-bit, oas 44-bit
(features 0x00000f0d)
[ 19.063087] arm-smmu-v3 400d0040000.smmu_alg: option
hisilicon,broken-prefetch-cmd
[ 19.071156] arm-smmu-v3 400d0040000.smmu_alg: ias 44-bit, oas 44-bit
(features 0x00000f0d)
[ 19.083316] arm-smmu-v3 408d0040000.smmu_alg: option
hisilicon,broken-prefetch-cmd
[ 19.091396] arm-smmu-v3 408d0040000.smmu_alg: ias 44-bit, oas 44-bit
(features 0x00000f0d)
[ 19.153965] loop: module loaded
[ 19.194531] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.201283] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.208022] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.214762] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.221502] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.228241] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.234981] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.241728] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.248466] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.255208] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.261947] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.268687] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.275425] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.282165] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.288903] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.295641] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.302383] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.309132] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.315871] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.322612] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.329350] cma: cma_alloc: alloc failed, req-size: 33 pages, ret: -12
[ 19.336254] cma: cma_alloc: alloc failed, req-size: 128 pages, ret: -12
[ 19.343315] cma: cma_alloc: alloc failed, req-size: 1024 pages, ret: -12
[ 19.353318] scsi host0: hisi_sas_v2_hw
[ 19.379826] phy-0:0: sas_phy_setup1 tc=0xffff801da2680630
[ 19.385424] sas_phy phy-0:0: sas_phy_setup2 tc=0xffff801da2680630
[ 19.391890] phy-0:1: sas_phy_setup1 tc=0xffff801da2680630
[ 19.397476] sas_phy phy-0:1: sas_phy_setup2 tc=0xffff801da2680630
[ 19.403918] phy-0:2: sas_phy_setup1 tc=0xffff801da2680630
[ 19.409503] sas_phy phy-0:2: sas_phy_setup2 tc=0xffff801da2680630
[ 19.415932] phy-0:3: sas_phy_setup1 tc=0xffff801da2680630
[ 19.421517] sas_phy phy-0:3: sas_phy_setup2 tc=0xffff801da2680630
[ 19.427945] phy-0:4: sas_phy_setup1 tc=0xffff801da2680630
[ 19.433530] sas_phy phy-0:4: sas_phy_setup2 tc=0xffff801da2680630
[ 19.439960] phy-0:5: sas_phy_setup1 tc=0xffff801da2680630
[ 19.445546] sas_phy phy-0:5: sas_phy_setup2 tc=0xffff801da2680630
[ 19.451974] phy-0:6: sas_phy_setup1 tc=0xffff801da2680630
[ 19.457558] sas_phy phy-0:6: sas_phy_setup2 tc=0xffff801da2680630
[ 19.464008] phy-0:7: sas_phy_setup1 tc=0xffff801da2680630
[ 19.469593] sas_phy phy-0:7: sas_phy_setup2 tc=0xffff801da2680630
[ 20.104417] hisi_sas_v2_hw a2000000.sas: phyup: phy0 link_rate=11
[ 20.110620] hisi_sas_v2_hw a2000000.sas: phyup: phy1 link_rate=11
[ 20.116788] hisi_sas_v2_hw a2000000.sas: phyup: phy2 link_rate=11
[ 20.122996] hisi_sas_v2_hw a2000000.sas: phyup: phy3 link_rate=11
[ 20.129158] hisi_sas_v2_hw a2000000.sas: phyup: phy4 link_rate=11
[ 20.133825] sas: phy-0:0 added to port-0:0, phy_mask:0x1
(500e004aaaaaaa1f)
[ 20.135321] hisi_sas_v2_hw a2000000.sas: phyup: phy5 link_rate=11
[ 20.141482] hisi_sas_v2_hw a2000000.sas: phyup: phy6 link_rate=11
[ 20.145678] sas: DOING DISCOVERY on port 0, pid:7
[ 20.147643] hisi_sas_v2_hw a2000000.sas: phyup: phy7 link_rate=11
[ 20.148243] hisi_sas_v2_hw a2000000.sas: dev[0:2] found
[ 20.159677] phy-0:0:0: sas_phy_setup1 tc=0xffff801da2680630
[ 20.165407] sas_phy phy-0:0:0: sas_phy_setup2 tc=0xffff801da2680630
[ 20.171945] sas: ex 500e004aaaaaaa1f phy00:U:0 attached:
0000000000000000 (no device)
[ 20.172090] phy-0:0:1: sas_phy_setup1 tc=0xffff801da2680630
[ 20.177820] sas_phy phy-0:0:1: sas_phy_setup2 tc=0xffff801da2680630
[ 20.184333] sas: ex 500e004aaaaaaa1f phy01:U:0 attached:
0000000000000000 (no device)
[ 20.184503] phy-0:0:2: sas_phy_setup1 tc=0xffff801da2680630
[ 20.190232] sas_phy phy-0:0:2: sas_phy_setup2 tc=0xffff801da2680630
[ 20.196743] sas: ex 500e004aaaaaaa1f phy02:U:A attached:
500e004aaaaaaa02 (stp)
[ 20.196911] phy-0:0:3: sas_phy_setup1 tc=0xffff801da2680630
[ 20.202639] sas_phy phy-0:0:3: sas_phy_setup2 tc=0xffff801da2680630
[ 20.209152] sas: ex 500e004aaaaaaa1f phy03:U:B attached:
5001e82002865f0e (ssp)
[ 20.209329] phy-0:0:4: sas_phy_setup1 tc=0xffff801da2680630
[ 20.215062] sas_phy phy-0:0:4: sas_phy_setup2 tc=0xffff801da2680630
[ 20.221576] sas: ex 500e004aaaaaaa1f phy04:U:B attached:
5001e8200287321e (ssp)
[ 20.221738] phy-0:0:5: sas_phy_setup1 tc=0xffff801da2680630
[ 20.227467] sas_phy phy-0:0:5: sas_phy_setup2 tc=0xffff801da2680630
[ 20.233976] sas: ex 500e004aaaaaaa1f phy05:U:B attached:
5000cca05b01d9c9 (ssp)
[ 20.234123] phy-0:0:6: sas_phy_setup1 tc=0xffff801da2680630
[ 20.239852] sas_phy phy-0:0:6: sas_phy_setup2 tc=0xffff801da2680630
[ 20.246366] sas: ex 500e004aaaaaaa1f phy06:U:0 attached:
0000000000000000 (no device)
[ 20.246535] phy-0:0:7: sas_phy_setup1 tc=0xffff801da2680630
[ 20.252268] sas_phy phy-0:0:7: sas_phy_setup2 tc=0xffff801da2680630
[ 20.258779] sas: ex 500e004aaaaaaa1f phy07:U:B attached:
5001e820028580ae (ssp)
[ 20.258945] phy-0:0:8: sas_phy_setup1 tc=0xffff801da2680630
[ 20.264674] sas_phy phy-0:0:8: sas_phy_setup2 tc=0xffff801da2680630
[ 20.271189] sas: ex 500e004aaaaaaa1f phy08:U:B attached:
5001e82002858496 (ssp)
[ 20.271361] phy-0:0:9: sas_phy_setup1 tc=0xffff801da2680630
[ 20.277093] sas_phy phy-0:0:9: sas_phy_setup2 tc=0xffff801da2680630
[ 20.283606] sas: ex 500e004aaaaaaa1f phy09:U:0 attached:
0000000000000000 (no device)
[ 20.283770] phy-0:0:10: sas_phy_setup1 tc=0xffff801da2680630
[ 20.289590] sas_phy phy-0:0:10: sas_phy_setup2 tc=0xffff801da2680630
[ 20.296195] sas: ex 500e004aaaaaaa1f phy10:U:B attached:
5001e82002862816 (ssp)
[ 20.296363] phy-0:0:11: sas_phy_setup1 tc=0xffff801da2680630
[ 20.302179] sas_phy phy-0:0:11: sas_phy_setup2 tc=0xffff801da2680630
[ 20.308785] sas: ex 500e004aaaaaaa1f phy11:U:B attached:
5001e8200285850a (ssp)
[ 20.308945] phy-0:0:12: sas_phy_setup1 tc=0xffff801da2680630
[ 20.314765] sas_phy phy-0:0:12: sas_phy_setup2 tc=0xffff801da2680630
[ 20.321362] sas: ex 500e004aaaaaaa1f phy12:U:0 attached:
0000000000000000 (no device)
[ 20.321518] phy-0:0:13: sas_phy_setup1 tc=0xffff801da2680630
[ 20.327338] sas_phy phy-0:0:13: sas_phy_setup2 tc=0xffff801da2680630
[ 20.333943] sas: ex 500e004aaaaaaa1f phy13:U:0 attached:
0000000000000000 (no device)
[ 20.334098] phy-0:0:14: sas_phy_setup1 tc=0xffff801da2680630
[ 20.339918] sas_phy phy-0:0:14: sas_phy_setup2 tc=0xffff801da2680630
[ 20.346520] sas: ex 500e004aaaaaaa1f phy14:U:0 attached:
0000000000000000 (no device)
[ 20.346677] phy-0:0:15: sas_phy_setup1 tc=0xffff801da2680630
[ 20.352497] sas_phy phy-0:0:15: sas_phy_setup2 tc=0xffff801da2680630
[ 20.359100] sas: ex 500e004aaaaaaa1f phy15:U:0 attached:
0000000000000000 (no device)
[ 20.359251] phy-0:0:16: sas_phy_setup1 tc=0xffff801da2680630
[ 20.365068] sas_phy phy-0:0:16: sas_phy_setup2 tc=0xffff801da2680630
[ 20.371672] sas: ex 500e004aaaaaaa1f phy16:U:B attached:
5001882016000000 (host)
[ 20.371838] phy-0:0:17: sas_phy_setup1 tc=0xffff801da2680630
[ 20.377659] sas_phy phy-0:0:17: sas_phy_setup2 tc=0xffff801da2680630
[ 20.384259] sas: ex 500e004aaaaaaa1f phy17:U:B attached:
5001882016000000 (host)
[ 20.384423] phy-0:0:18: sas_phy_setup1 tc=0xffff801da2680630
[ 20.390243] sas_phy phy-0:0:18: sas_phy_setup2 tc=0xffff801da2680630
[ 20.396850] sas: ex 500e004aaaaaaa1f phy18:U:B attached:
5001882016000000 (host)
[ 20.397017] phy-0:0:19: sas_phy_setup1 tc=0xffff801da2680630
[ 20.402836] sas_phy phy-0:0:19: sas_phy_setup2 tc=0xffff801da2680630
[ 20.409437] sas: ex 500e004aaaaaaa1f phy19:U:B attached:
5001882016000000 (host)
[ 20.409602] phy-0:0:20: sas_phy_setup1 tc=0xffff801da2680630
[ 20.415418] sas_phy phy-0:0:20: sas_phy_setup2 tc=0xffff801da2680630
[ 20.422022] sas: ex 500e004aaaaaaa1f phy20:U:B attached:
5001882016000000 (host)
[ 20.422173] phy-0:0:21: sas_phy_setup1 tc=0xffff801da2680630
[ 20.427994] sas_phy phy-0:0:21: sas_phy_setup2 tc=0xffff801da2680630
[ 20.434594] sas: ex 500e004aaaaaaa1f phy21:U:B attached:
5001882016000000 (host)
[ 20.434758] phy-0:0:22: sas_phy_setup1 tc=0xffff801da2680630
[ 20.440579] sas_phy phy-0:0:22: sas_phy_setup2 tc=0xffff801da2680630
[ 20.447185] sas: ex 500e004aaaaaaa1f phy22:U:B attached:
5001882016000000 (host)
[ 20.447364] phy-0:0:23: sas_phy_setup1 tc=0xffff801da2680630
[ 20.453184] sas_phy phy-0:0:23: sas_phy_setup2 tc=0xffff801da2680630
[ 20.459792] sas: ex 500e004aaaaaaa1f phy23:U:B attached:
5001882016000000 (host)
[ 20.459926] phy-0:0:24: sas_phy_setup1 tc=0xffff801da2680630
[ 20.465742] sas_phy phy-0:0:24: sas_phy_setup2 tc=0xffff801da2680630
[ 20.472346] sas: ex 500e004aaaaaaa1f phy24:D:B attached:
500e004aaaaaaa1e (host+target)
[ 20.473130] hisi_sas_v2_hw a2000000.sas: dev[2:5] found
[ 20.485177] hisi_sas_v2_hw a2000000.sas: dev[1:1] found
[ 20.497191] hisi_sas_v2_hw a2000000.sas: dev[3:1] found
[ 20.509241] hisi_sas_v2_hw a2000000.sas: dev[4:1] found
[ 20.514767] hisi_sas_v2_hw a2000000.sas: dev[5:1] found
[ 20.526781] hisi_sas_v2_hw a2000000.sas: dev[6:1] found
[ 20.538824] hisi_sas_v2_hw a2000000.sas: dev[7:1] found
[ 20.550846] hisi_sas_v2_hw a2000000.sas: dev[8:1] found
[ 20.563053] hisi_sas_v2_hw a2000000.sas: dev[9:1] found
[ 20.568532] sas: Enter sas_scsi_recover_host busy: 0 failed: 0
[ 20.568552] sas: ata1: end_device-0:0:2: dev error handler
[ 20.568564] sas: ata1: end_device-0:0:2: Unable to reset ata device?
[ 20.727878] ata1.00: ATA-8: HGST HUS724040ALA640, MFAOA8B0, max UDMA/133
[ 20.734666] ata1.00: 7814037168 sectors, multi 0: LBA48 NCQ (depth 32)
[ 20.743376] ata1.00: configured for UDMA/133
[ 20.747715] sas: --- Exit sas_scsi_recover_host: busy: 0 failed: 0
tries: 1
[ 20.759480] scsi 0:0:0:0: Direct-Access ATA HGST HUS724040AL
A8B0 PQ: 0 ANSI: 5
[ 20.768482] sd 0:0:0:0: [sda] 7814037168 512-byte logical blocks:
(4.00 TB/3.64 TiB)
[ 20.776358] sd 0:0:0:0: [sda] Write Protect is off
[ 20.781226] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[ 20.781285] sd 0:0:0:0: [sda] Write cache: enabled, read cache:
enabled, doesn't support DPO or FUA
[ 20.807142] scsi 0:0:1:0: Direct-Access SanDisk LT0200MO
P404 PQ: 0 ANSI: 6
[ 20.836050] sda:
[ 20.841757] sd 0:0:0:0: [sda] Attached SCSI disk
[ 21.106218] scsi 0:0:2:0: Direct-Access SanDisk LT0200MO
P404 PQ: 0 ANSI: 6
[ 21.283134] sd 0:0:1:0: [sdb] 390721968 512-byte logical blocks: (200
GB/186 GiB)
[ 21.325155] sd 0:0:1:0: [sdb] Write Protect is off
[ 21.330011] sd 0:0:1:0: [sdb] Mode Sense: cf 00 10 08
[ 21.368679] scsi 0:0:3:0: Direct-Access HGST HUC156030CSS200
A3F0 PQ: 0 ANSI: 6
[ 21.379727] sd 0:0:3:0: [sdd] 586072368 512-byte logical blocks: (300
GB/279 GiB)
[ 21.388536] sd 0:0:3:0: [sdd] Write Protect is off
[ 21.393388] sd 0:0:3:0: [sdd] Mode Sense: f7 00 10 08
[ 21.395700] sd 0:0:3:0: [sdd] Write cache: enabled, read cache:
enabled, supports DPO and FUA
[ 21.407151] sd 0:0:1:0: [sdb] Write cache: disabled, read cache:
disabled, supports DPO and FUA
[ 21.417692] scsi 0:0:4:0: Direct-Access SanDisk LT0200MO
P404 PQ: 0 ANSI: 6
[ 21.417958] sdd:
[ 21.439299] sd 0:0:3:0: [sdd] Attached SCSI disk
[ 21.582299] sd 0:0:2:0: [sdc] 390721968 512-byte logical blocks: (200
GB/186 GiB)
[ 21.624339] sd 0:0:2:0: [sdc] Write Protect is off
[ 21.629190] sd 0:0:2:0: [sdc] Mode Sense: cf 00 10 08
[ 21.706374] sd 0:0:2:0: [sdc] Write cache: disabled, read cache:
disabled, supports DPO and FUA
[ 21.716827] scsi 0:0:5:0: Direct-Access SanDisk LT0200MO
P404 PQ: 0 ANSI: 6
[ 21.893654] sd 0:0:4:0: [sde] 390721968 512-byte logical blocks: (200
GB/186 GiB)
[ 21.935677] sd 0:0:4:0: [sde] Write Protect is off
[ 21.940528] sd 0:0:4:0: [sde] Mode Sense: cf 00 10 08
[ 22.016279] scsi 0:0:6:0: Direct-Access SanDisk LT0200MO
P404 PQ: 0 ANSI: 6
[ 22.017886] sd 0:0:4:0: [sde] Write cache: disabled, read cache:
disabled, supports DPO and FUA
[ 22.069654] sdb: sdb1 sdb2
[ 22.192873] sd 0:0:5:0: [sdf] 390721968 512-byte logical blocks: (200
GB/186 GiB)
[ 22.234896] sd 0:0:5:0: [sdf] Write Protect is off
[ 22.239766] sd 0:0:5:0: [sdf] Mode Sense: cf 00 10 08
[ 22.315387] scsi 0:0:7:0: Direct-Access SanDisk LT0200MO
P404 PQ: 0 ANSI: 6
[ 22.316918] sd 0:0:5:0: [sdf] Write cache: disabled, read cache:
disabled, supports DPO and FUA
[ 22.323315] random: fast init done
[ 22.369160] sdc: sdc1 sdc2 sdc3
[ 22.492220] sd 0:0:6:0: [sdg] 390721968 512-byte logical blocks: (200
GB/186 GiB)
[ 22.534258] sd 0:0:6:0: [sdg] Write Protect is off
[ 22.539109] sd 0:0:6:0: [sdg] Mode Sense: cf 00 10 08
[ 22.577965] scsi 0:0:8:0: Enclosure 12G SAS Expander
RevB PQ: 0 ANSI: 6
[ 22.586309] scsi 0:0:8:0: Power-on or device reset occurred
[ 22.593121] sas: DONE DISCOVERY on port 0, pid:7, result:0
[ 22.593146] sas: broadcast received: 0
[ 22.593152] sas: broadcast received: 0
[ 22.593156] sas: broadcast received: 0
[ 22.593160] sas: broadcast received: 0
[ 22.593164] sas: broadcast received: 0
[ 22.593167] sas: broadcast received: 0
[ 22.593171] sas: broadcast received: 0
[ 22.593178] sas: phy1 matched wide port0
[ 22.593193] sas: phy-0:1 added to port-0:0, phy_mask:0x3
(500e004aaaaaaa1f)
[ 22.593219] sas: phy2 matched wide port0
[ 22.593232] sas: phy-0:2 added to port-0:0, phy_mask:0x7
(500e004aaaaaaa1f)
[ 22.593251] sas: phy3 matched wide port0
[ 22.593266] sas: phy-0:3 added to port-0:0, phy_mask:0xf
(500e004aaaaaaa1f)
[ 22.593285] sas: phy4 matched wide port0
[ 22.593300] sas: phy-0:4 added to port-0:0, phy_mask:0x1f
(500e004aaaaaaa1f)
[ 22.593319] sas: phy5 matched wide port0
[ 22.593332] sas: phy-0:5 added to port-0:0, phy_mask:0x3f
(500e004aaaaaaa1f)
[ 22.593349] sas: phy6 matched wide port0
[ 22.593363] sas: phy-0:6 added to port-0:0, phy_mask:0x7f
(500e004aaaaaaa1f)
[ 22.593381] sas: phy7 matched wide port0
[ 22.593394] sas: phy-0:7 added to port-0:0, phy_mask:0xff
(500e004aaaaaaa1f)
[ 22.616276] sd 0:0:6:0: [sdg] Write cache: disabled, read cache:
disabled, supports DPO and FUA
[ 22.641826] sd 0:0:1:0: [sdb] Attached SCSI disk
[ 22.681015] sde: sde1 sde2 sde3
[ 22.724602] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
[ 22.738490] 602b0000.uart: ttyAMA0 at MMIO 0x602b0000 (irq = 147,
base_baud = 0) is a SBSA
[ 22.746933] printk: console [ttyAMA0] enabled
[ 22.755712] printk: bootconsole [pl11] disabled
[ 22.791459] sd 0:0:7:0: [sdh] 390721968 512-byte logical blocks: (200
GB/186 GiB)
[ 22.828764] rtc-efi rtc-efi: setting system clock to 2018-11-07
18:32:59 UTC (1541615579)
[ 22.833477] sd 0:0:7:0: [sdh] Write Protect is off
[ 22.841725] sd 0:0:7:0: [sdh] Mode Sense: cf 00 10 08
[ 22.915642] sd 0:0:7:0: [sdh] Write cache: disabled, read cache:
disabled, supports DPO and FUA
[ 22.941208] sd 0:0:2:0: [sdc] Attached SCSI disk
[ 22.979798] sdf: sdf1 sdf2 sdf3
[ 23.253026] sd 0:0:4:0: [sde] Attached SCSI disk
[ 23.279081] sdg: sdg1 sdg2
[ 23.551508] sd 0:0:5:0: [sdf] Attached SCSI disk
[ 23.578883] sdh: sdh1 sdh2
[ 23.851190] sd 0:0:6:0: [sdg] Attached SCSI disk
[ 24.150829] sd 0:0:7:0: [sdh] Attached SCSI disk
[ 24.155934] Freeing unused kernel memory: 1408K
[ 24.160715] Run /init as init process
EOM
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2018-11-07 18:42 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-10-23 10:30 Crash report: Broken NUMA distance map causes crash on arm64 system John Garry
2018-10-25 11:01 ` John Garry
2018-10-30 9:26 ` Peter Zijlstra
2018-10-30 9:55 ` John Garry
2018-10-30 15:35 ` John Garry
2018-10-31 20:46 ` Peter Zijlstra
2018-11-01 10:01 ` John Garry
2018-11-02 9:39 ` Peter Zijlstra
2018-11-02 10:10 ` John Garry
2018-11-02 10:50 ` Peter Zijlstra
2018-11-02 12:08 ` John Garry
2018-11-02 12:19 ` Peter Zijlstra
2018-11-07 18:42 ` John Garry
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).