* aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
@ 2016-10-13 22:50 Laszlo Ersek
2016-10-14 8:05 ` Andrew Jones
0 siblings, 1 reply; 11+ messages in thread
From: Laszlo Ersek @ 2016-10-13 22:50 UTC (permalink / raw)
To: Zhen Lei, Will Deacon
Cc: main kernel list, linux-arm-kernel, Ard Biesheuvel, Shannon Zhao,
Drew Jones, Wei Huang
Hi,
the following regression is experienced in aarch64 qemu/KVM virtual
machines, using the ArmVirtQemu virtual UEFI firmware platform built
from edk2 (EFI Development Kit II).
(1) When booting current master (b67be92feb48) or the bisected first bad
commit (7ba5f605f3a0) with DT enabled, everything works fine.
(2) When booting the above two commits with DT disabled -- meaning:
either the firmware provides ACPI only and no DT, or "acpi=force" is
passed on the kernel command line --, the boot breaks with one of
the following symptoms:
(2a) no messages are printed after
> EFI stub: Booting Linux Kernel...
> ConvertPages: Incompatible memory types
> EFI stub: Using DTB from configuration table
> EFI stub: Exiting boot services and installing virtual address map...
and the kernel seems to be spinning in an infinite loop (no
messages despite "earlycon" and friends),
(2b) or the following crash dump is printed:
> EFI stub: Booting Linux Kernel...
> ConvertPages: Incompatible memory types
> EFI stub: Using DTB from configuration table
> EFI stub: Exiting boot services and installing virtual address map...
> Booting Linux on physical CPU 0x0
> Linux version 4.8.0-rc3+ (root@aarch64-vgpu-1) (gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC) ) #19 SMP Thu Oct 13 22:30:27 CEST 2016
> Boot CPU: AArch64 Processor [500f0000]
> earlycon: pl11 at MMIO 0x0000000009000000 (options '')
> bootconsole [pl11] enabled
> debug: ignoring loglevel setting.
> efi: Getting EFI parameters from FDT:
> efi: EFI v2.60 by EDK II
> efi: SMBIOS 3.0=0x23bdb0000 ACPI 2.0=0x2386d0000 MEMATTR=0x23a665018
> cma: Reserved 512 MiB at 0x00000000c0000000
> ACPI: Early table checksum verification disabled
> ACPI: RSDP 0x00000002386D0000 000024 (v02 BOCHS )
> ACPI: XSDT 0x00000002386C0000 00004C (v01 BOCHS BXPCFACP 00000001 01000013)
> ACPI: FACP 0x0000000238410000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001)
> ACPI: DSDT 0x0000000238420000 001159 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001)
> ACPI: APIC 0x0000000238400000 0002BC (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001)
> ACPI: GTDT 0x00000002383F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001)
> ACPI: MCFG 0x00000002383E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001)
> ACPI: SPCR 0x00000002383D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001)
> ACPI: NUMA: Failed to initialise from firmware
> NUMA: Faking a node at [mem 0x0000000000000000-0x000000023fffffff]
> NUMA: Adding memblock [0x40000000 - 0xfffeffff] on node 0
> NUMA: Adding memblock [0xffff0000 - 0xffffffff] on node 0
> NUMA: Adding memblock [0x100000000 - 0x2383cffff] on node 0
> NUMA: Adding memblock [0x2383d0000 - 0x23874ffff] on node 0
> NUMA: Adding memblock [0x238750000 - 0x23bc1ffff] on node 0
> NUMA: Adding memblock [0x23bc20000 - 0x23bffffff] on node 0
> NUMA: Adding memblock [0x23c000000 - 0x23fffffff] on node 0
> NUMA: Initmem setup node 0 [mem 0x40000000-0x23fffffff]
> NUMA: NODE_DATA [mem 0x23fff2580-0x23fffffff]
> Zone ranges:
> DMA [mem 0x0000000040000000-0x00000000ffffffff]
> Normal [mem 0x0000000100000000-0x000000023fffffff]
> Movable zone start for each node
> Early memory node ranges
> node 0: [mem 0x0000000040000000-0x00000000fffeffff]
> node 0: [mem 0x00000000ffff0000-0x00000000ffffffff]
> node 0: [mem 0x0000000100000000-0x00000002383cffff]
> node 0: [mem 0x00000002383d0000-0x000000023874ffff]
> node 0: [mem 0x0000000238750000-0x000000023bc1ffff]
> node 0: [mem 0x000000023bc20000-0x000000023bffffff]
> node 0: [mem 0x000000023c000000-0x000000023fffffff]
> Initmem setup node 0 [mem 0x0000000040000000-0x000000023fffffff]
> On node 0 totalpages: 131072
> DMA zone: 48 pages used for memmap
> DMA zone: 0 pages reserved
> DMA zone: 49152 pages, LIFO batch:1
> Normal zone: 80 pages used for memmap
> Normal zone: 81920 pages, LIFO batch:1
> psci: probing for conduit method from ACPI.
> psci: PSCIv0.2 detected in firmware.
> psci: Using standard PSCI v0.2 function IDs
> psci: Trusted OS migration not required
> percpu: Embedded 3 pages/cpu @fffffe01ffdb0000 s117320 r8192 d71096 u196608
> pcpu-alloc: s117320 r8192 d71096 u196608 alloc=3*65536
> pcpu-alloc: [0] 0 [1] 1 [2] 2 [3] 3 [4] 4 [5] 5 [6] 6 [7] 7
> Detected PIPT I-cache on CPU0
> Built 1 zonelists in Node order, mobility grouping on. Total pages: 130944
> Policy zone: Normal
> Kernel command line: BOOT_IMAGE=/vmlinuz-4.8.0-rc3+ root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap console=ttyAMA0 earlyprintk=pl011,0x9000000 earlycon ignore_loglevel LANG=en_US.UTF-8 acpi=force
> PID hash table entries: 4096 (order: -1, 32768 bytes)
> software IO TLB [mem 0xfbfe0000-0xfffe0000] (64MB) mapped at [fffffe00bbfe0000-fffffe00bffdffff]
> Memory: 7717184K/8388608K available (8956K kernel code, 1564K rwdata, 3712K rodata, 1536K init, 15875K bss, 147136K reserved, 524288K cma-reserved)
> Virtual kernel memory layout:
> modules : 0xfffffc0000000000 - 0xfffffc0008000000 ( 128 MB)
> vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000 ( 2045 GB)
> .text : 0xfffffc0008080000 - 0xfffffc0008940000 ( 8960 KB)
> .rodata : 0xfffffc0008940000 - 0xfffffc0008cf0000 ( 3776 KB)
> .init : 0xfffffc0008cf0000 - 0xfffffc0008e70000 ( 1536 KB)
> .data : 0xfffffc0008e70000 - 0xfffffc0008ff7200 ( 1565 KB)
> .bss : 0xfffffc0008ff7200 - 0xfffffc0009f78120 ( 15876 KB)
> fixed : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000 ( 4288 KB)
> PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000 ( 16 MB)
> vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000 ( 2 GB maximum)
> 0xfffffdff80000000 - 0xfffffdff80800000 ( 8 MB actual)
> memory : 0xfffffe0000000000 - 0xfffffe0200000000 ( 8192 MB)
> SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=1
> Running RCU self tests
> Hierarchical RCU implementation.
> RCU lockdep checking is enabled.
> Build-time adjustment of leaf fanout to 64.
> RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=8.
> RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=8
> kmemleak: Kernel memory leak detector disabled
> NR_IRQS:64 nr_irqs:64 0
> GICv2m: ACPI overriding V2M MSI_TYPER (base:80, num:64)
> GICv2m: range[mem 0x08020000-0x08020fff], SPI[80:143]
> GIC: PPI11 is secure or misconfigured
> arm_arch_timer: WARNING: Invalid trigger for IRQ3, assuming level low
> arm_arch_timer: WARNING: Please fix your firmware
> arm_arch_timer: Architected cp15 timer(s) running at 50.00MHz (virt).
> clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
> sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
> Console: colour dummy device 80x25
> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> ... MAX_LOCKDEP_SUBCLASSES: 8
> ... MAX_LOCK_DEPTH: 48
> ... MAX_LOCKDEP_KEYS: 8191
> ... CLASSHASH_SIZE: 4096
> ... MAX_LOCKDEP_ENTRIES: 32768
> ... MAX_LOCKDEP_CHAINS: 65536
> ... CHAINHASH_SIZE: 32768
> memory used by lock dependency info: 8159 kB
> per task-struct memory footprint: 1920 bytes
> Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=50000)
> pid_max: default: 32768 minimum: 301
> ACPI: Core revision 20160422
> ACPI: 1 ACPI AML tables successfully acquired and loaded
>
> Security Framework initialized
> Yama: becoming mindful.
> SELinux: Initializing.
> SELinux: Starting in permissive mode
> Dentry cache hash table entries: 1048576 (order: 7, 8388608 bytes)
> Inode-cache hash table entries: 524288 (order: 6, 4194304 bytes)
> Mount-cache hash table entries: 16384 (order: 1, 131072 bytes)
> Mountpoint-cache hash table entries: 16384 (order: 1, 131072 bytes)
> ftrace: allocating 28918 entries in 8 pages
> ASID allocator initialised with 65536 entries
> Unable to handle kernel paging request at virtual address e18000009518
> pgd = fffffc0009fb0000
> [e18000009518] *pgd=0000000000000000, *pud=0000000000000000, *pmd=0000000000000000
> Internal error: Oops: 96000004 [#1] SMP
> Modules linked in:
> CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.8.0-rc3+ #19
> Hardware name: linux,dummy-virt (DT)
> task: fffffe01dc07b600 task.stack: fffffe01fa680000
> PC is at __ll_sc_atomic_add+0x20/0x40
> LR is at __lock_acquire+0xe8/0x698
> pc : [<fffffc000847a9a8>] lr : [<fffffc00081357e8>] pstate: 800000c5
> sp : fffffe01fa6838c0
> x29: fffffe01fa6838c0 x28: fffffc0008ea3000
> x27: fffffc0008ea2358 x26: fffffc0009c84000
> x25: 0000000000000001 x24: 0000000000000000
> x23: fffffe01dc07b600 x22: 0000000000000000
> x21: fffffe01ffd80818 x20: 0000000000000000
> x19: fffffe01ffd80818 x18: 0000000000000000
> x17: 0000000000000000 x16: 0000000000000000
> x15: ffffffffffffffff x14: 00000000000002b7
> x13: 00000000000002b7 x12: 0000000000000038
> x11: 0000000000000005 x10: 0101010101010101
> x9 : 0000000000000001 x8 : 0000e18000009518
> x7 : fffffc000829124c x6 : 0000000000000000
> x5 : 0000000000000080 x4 : 0000e18000009380
> x3 : 0000000000000000 x2 : 000072000000c380
> x1 : 0000e18000009518 x0 : fffffc00081357e8
>
> Process swapper/0 (pid: 1, stack limit = 0xfffffe01fa680020)
> Stack: (0xfffffe01fa6838c0 to 0xfffffe01fa684000)
> 38c0: fffffe01fa6838e0 fffffc00081357e8 fffffe01fa680000 0000000000000001
> 38e0: fffffe01fa683960 fffffc0008136170 fffffe01ffd80818 0000000000000000
> 3900: 0000000000000000 0000000000000000 0000000000000001 0000000000000000
> 3920: fffffc000829124c 00000000000000c0 fffffc0008ea2358 fffffc0008ea3000
> 3940: fffffc000812dde0 00000000000000c0 fffffc0000000000 fffffc0000000000
> 3960: fffffe01fa6839d0 fffffc000892c9ac fffffe01ffd80800 fffffc000829124c
> 3980: fffffe01ffd80800 fffffc000829200c fffffe01f401fc00 000000000000e8e8
> 39a0: fffffe01f401fc00 fffffe01f401fcf8 fffffe01fff1ea90 0000000000000000
> 39c0: fffffe01dc07b680 fffffc0008ea2000 fffffe01fa6839f0 fffffc000829124c
> 39e0: 00000000ffffffff fffffe01ffd80800 fffffe01fa683b10 fffffc0008291ce8
> 3a00: 00000000ffffffff 0000000000000001 00000000024080c0 fffffc000829200c
> 3a20: 0000000000210d00 000000000000e8e8 fffffe01f401fc00 fffffe01f401fcf8
> 3a40: fffffe01fff1ea90 0000000000000000 fffffe01fa683a80 fffffc0008088954
> 3a60: fffffc0009048eb8 fffffe01dc07b600 fffffe01dc07be60 fffffe01fff1eaa0
> 3a80: fffffe01fa683ad0 fffffc00024080c0 fffffc0009048eb8 fffffc00095a3000
> 3aa0: fffffc0008132e80 fffffc0009048eb8 0000000000000000 0000000000000000
> 3ac0: fffffe01fa683eb0 fffffc0008083330 fffffe01fa683b10 fffffc0008291b18
> 3ae0: 7f7f7f7f7f7f7f7f ff1f2877372f2427 0101010101010101 0000000000000005
> 3b00: 0000000000000038 0000000000000000 fffffe01fa683c30 fffffc000829200c
> 3b20: 0000000000000040 fffffe01f401fc00 00000000024080c0 00000000ffffffff
> 3b40: fffffc00084ac9f0 fffffe01fff1ea90 0000000000000000 0000000000000006
> 3b60: 0000000000000043 fffffc0008b89670 fffffc0008f374b8 0000000000000000
> 3b80: fffffe01fa683bd0 fffffc0008135200 fffffe01fff1eaa0 0000000000000000
> 3ba0: fffffe0100000000 fffffc00084ac9f0 fffffe01fa683bf0 fffffc0008131904
> 3bc0: fffffe01fa680000 0000000102000200 fffffc0008bb4cf0 0000000000000189
> 3be0: 0000000000000028 fffffe01dc07b600 fffffe01fa683c10 fffffc00080fff1c
> 3c00: fffffc0008fbb39e 0000000000000000 fffffe01fa683c40 fffffc0008100034
> 3c20: fffffe01fa683c30 fffffc0008291ff4 fffffe01fa683c70 fffffc00082927dc
> 3c40: fffffe01f401fc00 00000000024080c0 fffffc00084ac9f0 fffffe01f401fc00
> 3c60: 0000000000000028 fffffc0008ea4000 fffffe01fa683cd0 fffffc00084ac9f0
> 3c80: fffffc0008ff6090 fffffc0008b89670 fffffc0008fd5f90 0000000000000002
> 3ca0: fffffc0008fd5f10 0000000000000003 00000000000001bd 0000000000000006
> 3cc0: 0000000000000043 fffffc0008b88a10 fffffe01fa683d10 fffffc0008d25090
> 3ce0: fffffc0008ff6090 fffffc0008fd5f90 fffffc0008fd5f90 fffffc0008fd5000
> 3d00: 0000000000000002 fffffc0008ea2398 fffffe01fa683d90 fffffc0008083594
> 3d20: fffffc0008d24fa0 fffffe01fa680000 0000000000000000 0000000000000000
> 3d40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3d60: 0000000000000000 0000000000000000 fffffe01fa683d90 fffffc0008b89d18
> 3d80: 000000000000000e 0000000000000019 fffffe01fa683e00 fffffc0008cf0d2c
> 3da0: fffffc0008e1c288 fffffc0008e1c2c8 0000000000000040 0000000000000000
> 3dc0: fffffe01fa683e00 fffffc0008cf0d1c fffffc0008e1c200 fffffc0008e1c2c8
> 3de0: 0000000000000040 0000000000000000 0000000000000000 fffffc0008e1c2c8
> 3e00: fffffe01fa683ea0 fffffc0008924978 fffffc0008924960 0000000000000000
> 3e20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3e40: 0000000000000000 0000000000000000 0000000000000000 0000000000000001
> 3e60: 0000000000000001 0000000000000000 0000000000000000 0000000000000000
> 3e80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3ea0: 0000000000000000 fffffc0008083330 fffffc0008924960 0000000000000000
> 3ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> 3fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
> 3fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> Call trace:
> Exception stack(0xfffffe01fa6836f0 to 0xfffffe01fa683820)
> 36e0: fffffe01ffd80818 0000040000000000
> 3700: fffffe01fa6838c0 fffffc000847a9a8 fffffe01fff1b580 fffffe01fff1b580
> 3720: fffffc0008927614 fffffc0008ea1000 fffffe01fa683740 00000000000000c0
> 3740: fffffe01fa683780 fffffc000811794c fffffe01fa6837e0 fffffc0008135a64
> 3760: 87cee53ad487914d fffffe01dc07be60 0000000000000001 0000000000000000
> 3780: fffffe01dc07b600 0000000000000000 fffffc00081357e8 0000e18000009518
> 37a0: 000072000000c380 0000000000000000 0000e18000009380 0000000000000080
> 37c0: 0000000000000000 fffffc000829124c 0000e18000009518 0000000000000001
> 37e0: 0101010101010101 0000000000000005 0000000000000038 00000000000002b7
> 3800: 00000000000002b7 ffffffffffffffff 0000000000000000 0000000000000000
> [<fffffc000847a9a8>] __ll_sc_atomic_add+0x20/0x40
> [<fffffc00081357e8>] __lock_acquire+0xe8/0x698
> [<fffffc0008136170>] lock_acquire+0xd8/0x2c0
> [<fffffc000892c9ac>] _raw_spin_lock+0x4c/0x60
> [<fffffc000829124c>] get_partial_node.isra.23+0x4c/0x440
> [<fffffc0008291ce8>] ___slab_alloc+0x438/0x708
> [<fffffc000829200c>] __slab_alloc+0x54/0xa0
> [<fffffc00082927dc>] kmem_cache_alloc_trace+0x35c/0x428
> [<fffffc00084ac9f0>] ddebug_add_module+0x38/0xf0
> [<fffffc0008d25090>] dynamic_debug_init+0xf0/0x2a0
> [<fffffc0008083594>] do_one_initcall+0x44/0x138
> [<fffffc0008cf0d2c>] kernel_init_freeable+0x17c/0x2e0
> [<fffffc0008924978>] kernel_init+0x18/0x110
> [<fffffc0008083330>] ret_from_fork+0x10/0x20
> Code: aa1e03e0 aa0103e8 d503201f f9800111 (885f7d00)
> ---[ end trace 0000000000000000 ]---
> note: swapper/0[1] exited with preempt_count 1
> Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>
> ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>
(3) Please find the bisection log below:
> git bisect start
> # bad: [b67be92feb486f800d80d72c67fd87b47b79b18e] Merge tag 'pwm/for-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/thierry.reding/linux-pwm
> git bisect bad b67be92feb486f800d80d72c67fd87b47b79b18e
> # good: [c8d2bc9bc39ebea8437fd974fdbc21847bb897a3] Linux 4.8
> git bisect good c8d2bc9bc39ebea8437fd974fdbc21847bb897a3
> # bad: [41844e36206be90cd4d962ea49b0abc3612a99d0] Merge tag 'staging-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/staging
> git bisect bad 41844e36206be90cd4d962ea49b0abc3612a99d0
> # bad: [d268dbe76a53d72cc41316eb59e7968db60e77ad] Merge tag 'pinctrl-v4.9-1' of git://git.kernel.org/pub/scm/linux/kernel/git/linusw/linux-pinctrl
> git bisect bad d268dbe76a53d72cc41316eb59e7968db60e77ad
> # bad: [02bafd96f3a5d8e610b19033ffec55b92459aaae] Merge tag 'docs-4.9' of git://git.lwn.net/linux
> git bisect bad 02bafd96f3a5d8e610b19033ffec55b92459aaae
> # bad: [9929780e86854833e649b39b290b5fe921eb1701] Merge tag 'driver-core-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/driver-core
> git bisect bad 9929780e86854833e649b39b290b5fe921eb1701
> # bad: [12b7bcb43e6ea834ab2f5dc52d971e379a0ca109] Merge branch 'perf-core-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
> git bisect bad 12b7bcb43e6ea834ab2f5dc52d971e379a0ca109
> # bad: [72d39926f098b0c4ad95e1461595a8d6d403c14d] Merge tag 'acpi-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
> git bisect bad 72d39926f098b0c4ad95e1461595a8d6d403c14d
> # bad: [72ec94560d7ee1d3a61d5904fd9a5bf68bf3b11a] Merge tag 'pm-4.9-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rafael/linux-pm
> git bisect bad 72ec94560d7ee1d3a61d5904fd9a5bf68bf3b11a
> # bad: [792d47379f4d4c76692f1795f33d38582f8907fa] arm64: alternative: add auto-nop infrastructure
> git bisect bad 792d47379f4d4c76692f1795f33d38582f8907fa
> # good: [dc00247576fdb97211e1959b4dfd2a7893cf9d0b] arm64: kernel: re-export _cpu_resume() from sleep.S
> git bisect good dc00247576fdb97211e1959b4dfd2a7893cf9d0b
> # good: [9787ed6e5cee7a62320f3014eb5e7b373502c292] of/numa: remove a duplicated warning
> git bisect good 9787ed6e5cee7a62320f3014eb5e7b373502c292
> # bad: [c47a1900ad710fd2c97127e2ba19da1df79cf733] arm64: Rearrange CPU errata workaround checks
> git bisect bad c47a1900ad710fd2c97127e2ba19da1df79cf733
> # good: [7af3a0a992524ffddc342cd1481cc4dcb3f1da71] arm64/numa: support HAVE_SETUP_PER_CPU_AREA
> git bisect good 7af3a0a992524ffddc342cd1481cc4dcb3f1da71
> # bad: [7ba5f605f3a0d9495aad539eeb8346d726dfc183] arm64/numa: remove the limitation that cpu0 must bind to node0
> git bisect bad 7ba5f605f3a0d9495aad539eeb8346d726dfc183
> # good: [df7ffa34cc0c06bfa7206732df78725ff34633ee] arm64/numa: remove some useless code
> git bisect good df7ffa34cc0c06bfa7206732df78725ff34633ee
> # first bad commit: [7ba5f605f3a0d9495aad539eeb8346d726dfc183] arm64/numa: remove the limitation that cpu0 must bind to node0
I repeatedly retested the first bad commit:
> commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
> Author: Zhen Lei <thunder.leizhen@huawei.com>
> Date: Thu Sep 1 14:55:04 2016 +0800
>
> arm64/numa: remove the limitation that cpu0 must bind to node0
and its direct ancestor:
> commit df7ffa34cc0c06bfa7206732df78725ff34633ee
> Author: Zhen Lei <thunder.leizhen@huawei.com>
> Date: Thu Sep 1 14:55:03 2016 +0800
>
> arm64/numa: remove some useless code
The offending commit consistently fails to boot with the described
symptoms when DT is disabled, and succeeds to boot when DT is enabled.
The predecessor commit consistently succeeds to boot regardless of DT
versus ACPI.
(4) Analysis (well, a lame attempt at that, because I have zero
familiarity with this code). Let me quote the patch:
> commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
> Author: Zhen Lei <thunder.leizhen@huawei.com>
> Date: Thu Sep 1 14:55:04 2016 +0800
>
> arm64/numa: remove the limitation that cpu0 must bind to node0
>
> 1. Remove the old binding code.
> 2. Read the nid of cpu0 from dts.
> 3. Fallback the nid of cpu0 to 0 when numa=off is set in bootargs.
>
> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> Signed-off-by: Will Deacon <will.deacon@arm.com>
>
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index c3c08368a685..8b048e6ec34a 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -624,6 +624,7 @@ static void __init of_parse_and_init_cpus(void)
> }
>
> bootcpu_valid = true;
> + early_map_cpu_to_node(0, of_node_to_nid(dn));
>
> /*
> * cpu_logical_map has already been
> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> index 0a15f010b64a..778a985c8a70 100644
> --- a/arch/arm64/mm/numa.c
> +++ b/arch/arm64/mm/numa.c
> @@ -116,16 +116,24 @@ static void __init setup_node_to_cpumask_map(void)
> */
> void numa_store_cpu_info(unsigned int cpu)
> {
> - map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
> + map_cpu_to_node(cpu, cpu_to_node_map[cpu]);
> }
>
> void __init early_map_cpu_to_node(unsigned int cpu, int nid)
> {
> /* fallback to node 0 */
> - if (nid < 0 || nid >= MAX_NUMNODES)
> + if (nid < 0 || nid >= MAX_NUMNODES || numa_off)
> nid = 0;
>
> cpu_to_node_map[cpu] = nid;
> +
> + /*
> + * We should set the numa node of cpu0 as soon as possible, because it
> + * has already been set up online before. cpu_to_node(0) will soon be
> + * called.
> + */
> + if (!cpu)
> + set_cpu_numa_node(cpu, nid);
> }
>
> #ifdef CONFIG_HAVE_SETUP_PER_CPU_AREA
> @@ -393,10 +401,6 @@ static int __init numa_init(int (*init_func)(void))
>
> setup_node_to_cpumask_map();
>
> - /* init boot processor */
> - cpu_to_node_map[0] = 0;
> - map_cpu_to_node(0, 0);
> -
> return 0;
> }
>
The commit message states that the numa-id (nid) for CPU#0 is read from
the DTS. If there is no DT, I think that won't work so well.
Second, the patch replaces the unconditional, static
CPU#0 <-> numa-node#0
mapping in numa_init(), which is independent of ACPI vs. DT, with a
DT-dependent early_map_cpu_to_node() call, in of_parse_and_init_cpus().
The ACPI branch is regressed by this, because on that branch we now
don't create a
CPU#0 <-> numa-node#whatever
mapping at all.
(5) The entry
ACPI: NUMA: Failed to initialise from firmware
in the dmesg doesn't imply an error in the firmware; it just means that
the firmware does not provide an SRAT table. That is valid if there's
only one NUMA node in the system.
(6) For reference, the MADT is:
> /*
> * Intel ACPI Component Architecture
> * AML/ASL+ Disassembler version 20160831-64
> * Copyright (c) 2000 - 2016 Intel Corporation
> *
> * Disassembly of apic.dat, Fri Oct 14 00:21:16 2016
> *
> * ACPI Data Table [APIC]
> *
> * Format: [HexOffset DecimalOffset ByteLength] FieldName : FieldValue
> */
>
> [000h 0000 4] Signature : "APIC" [Multiple APIC Description Table (MADT)]
> [004h 0004 4] Table Length : 000002BC
> [008h 0008 1] Revision : 03
> [009h 0009 1] Checksum : 18
> [00Ah 0010 6] Oem ID : "BOCHS "
> [010h 0016 8] Oem Table ID : "BXPCAPIC"
> [018h 0024 4] Oem Revision : 00000001
> [01Ch 0028 4] Asl Compiler ID : "BXPC"
> [020h 0032 4] Asl Compiler Revision : 00000001
>
> [024h 0036 4] Local Apic Address : 00000000
> [028h 0040 4] Flags (decoded below) : 00000000
> PC-AT Compatibility : 0
>
> [02Ch 0044 1] Subtable Type : 0C [Generic Interrupt Distributor]
> [02Dh 0045 1] Length : 18
> [02Eh 0046 2] Reserved : 0000
> [030h 0048 4] Local GIC Hardware ID : 00000000
> [034h 0052 8] Base Address : 0000000008000000
> [03Ch 0060 4] Interrupt Base : 00000000
> [040h 0064 1] Version : 02
> [041h 0065 3] Reserved : 000000
>
> [044h 0068 1] Subtable Type : 0B [Generic Interrupt Controller]
> [045h 0069 1] Length : 4C
> [046h 0070 2] Reserved : 0000
> [048h 0072 4] CPU Interface Number : 00000000
> [04Ch 0076 4] Processor UID : 00000000
> [050h 0080 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [054h 0084 4] Parking Protocol Version : 00000000
> [058h 0088 4] Performance Interrupt : 00000017
> [05Ch 0092 8] Parked Address : 0000000000000000
> [064h 0100 8] Base Address : 0000000008010000
> [06Ch 0108 8] Virtual GIC Base Address : 0000000000000000
> [074h 0116 8] Hypervisor GIC Base Address : 0000000000000000
> [07Ch 0124 4] Virtual GIC Interrupt : 00000000
> [080h 0128 8] Redistributor Base Address : 0000000000000000
> [088h 0136 8] ARM MPIDR : 0000000000000000
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [090h 0144 1] Subtable Type : 0B [Generic Interrupt Controller]
> [091h 0145 1] Length : 4C
> [092h 0146 2] Reserved : 0000
> [094h 0148 4] CPU Interface Number : 00000001
> [098h 0152 4] Processor UID : 00000001
> [09Ch 0156 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [0A0h 0160 4] Parking Protocol Version : 00000000
> [0A4h 0164 4] Performance Interrupt : 00000017
> [0A8h 0168 8] Parked Address : 0000000000000000
> [0B0h 0176 8] Base Address : 0000000008010000
> [0B8h 0184 8] Virtual GIC Base Address : 0000000000000000
> [0C0h 0192 8] Hypervisor GIC Base Address : 0000000000000000
> [0C8h 0200 4] Virtual GIC Interrupt : 00000000
> [0CCh 0204 8] Redistributor Base Address : 0000000000000000
> [0D4h 0212 8] ARM MPIDR : 0000000000000001
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [0DCh 0220 1] Subtable Type : 0B [Generic Interrupt Controller]
> [0DDh 0221 1] Length : 4C
> [0DEh 0222 2] Reserved : 0000
> [0E0h 0224 4] CPU Interface Number : 00000002
> [0E4h 0228 4] Processor UID : 00000002
> [0E8h 0232 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [0ECh 0236 4] Parking Protocol Version : 00000000
> [0F0h 0240 4] Performance Interrupt : 00000017
> [0F4h 0244 8] Parked Address : 0000000000000000
> [0FCh 0252 8] Base Address : 0000000008010000
> [104h 0260 8] Virtual GIC Base Address : 0000000000000000
> [10Ch 0268 8] Hypervisor GIC Base Address : 0000000000000000
> [114h 0276 4] Virtual GIC Interrupt : 00000000
> [118h 0280 8] Redistributor Base Address : 0000000000000000
> [120h 0288 8] ARM MPIDR : 0000000000000002
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [128h 0296 1] Subtable Type : 0B [Generic Interrupt Controller]
> [129h 0297 1] Length : 4C
> [12Ah 0298 2] Reserved : 0000
> [12Ch 0300 4] CPU Interface Number : 00000003
> [130h 0304 4] Processor UID : 00000003
> [134h 0308 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [138h 0312 4] Parking Protocol Version : 00000000
> [13Ch 0316 4] Performance Interrupt : 00000017
> [140h 0320 8] Parked Address : 0000000000000000
> [148h 0328 8] Base Address : 0000000008010000
> [150h 0336 8] Virtual GIC Base Address : 0000000000000000
> [158h 0344 8] Hypervisor GIC Base Address : 0000000000000000
> [160h 0352 4] Virtual GIC Interrupt : 00000000
> [164h 0356 8] Redistributor Base Address : 0000000000000000
> [16Ch 0364 8] ARM MPIDR : 0000000000000003
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [174h 0372 1] Subtable Type : 0B [Generic Interrupt Controller]
> [175h 0373 1] Length : 4C
> [176h 0374 2] Reserved : 0000
> [178h 0376 4] CPU Interface Number : 00000004
> [17Ch 0380 4] Processor UID : 00000004
> [180h 0384 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [184h 0388 4] Parking Protocol Version : 00000000
> [188h 0392 4] Performance Interrupt : 00000017
> [18Ch 0396 8] Parked Address : 0000000000000000
> [194h 0404 8] Base Address : 0000000008010000
> [19Ch 0412 8] Virtual GIC Base Address : 0000000000000000
> [1A4h 0420 8] Hypervisor GIC Base Address : 0000000000000000
> [1ACh 0428 4] Virtual GIC Interrupt : 00000000
> [1B0h 0432 8] Redistributor Base Address : 0000000000000000
> [1B8h 0440 8] ARM MPIDR : 0000000000000004
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [1C0h 0448 1] Subtable Type : 0B [Generic Interrupt Controller]
> [1C1h 0449 1] Length : 4C
> [1C2h 0450 2] Reserved : 0000
> [1C4h 0452 4] CPU Interface Number : 00000005
> [1C8h 0456 4] Processor UID : 00000005
> [1CCh 0460 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [1D0h 0464 4] Parking Protocol Version : 00000000
> [1D4h 0468 4] Performance Interrupt : 00000017
> [1D8h 0472 8] Parked Address : 0000000000000000
> [1E0h 0480 8] Base Address : 0000000008010000
> [1E8h 0488 8] Virtual GIC Base Address : 0000000000000000
> [1F0h 0496 8] Hypervisor GIC Base Address : 0000000000000000
> [1F8h 0504 4] Virtual GIC Interrupt : 00000000
> [1FCh 0508 8] Redistributor Base Address : 0000000000000000
> [204h 0516 8] ARM MPIDR : 0000000000000005
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [20Ch 0524 1] Subtable Type : 0B [Generic Interrupt Controller]
> [20Dh 0525 1] Length : 4C
> [20Eh 0526 2] Reserved : 0000
> [210h 0528 4] CPU Interface Number : 00000006
> [214h 0532 4] Processor UID : 00000006
> [218h 0536 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [21Ch 0540 4] Parking Protocol Version : 00000000
> [220h 0544 4] Performance Interrupt : 00000017
> [224h 0548 8] Parked Address : 0000000000000000
> [22Ch 0556 8] Base Address : 0000000008010000
> [234h 0564 8] Virtual GIC Base Address : 0000000000000000
> [23Ch 0572 8] Hypervisor GIC Base Address : 0000000000000000
> [244h 0580 4] Virtual GIC Interrupt : 00000000
> [248h 0584 8] Redistributor Base Address : 0000000000000000
> [250h 0592 8] ARM MPIDR : 0000000000000006
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [258h 0600 1] Subtable Type : 0B [Generic Interrupt Controller]
> [259h 0601 1] Length : 4C
> [25Ah 0602 2] Reserved : 0000
> [25Ch 0604 4] CPU Interface Number : 00000007
> [260h 0608 4] Processor UID : 00000007
> [264h 0612 4] Flags (decoded below) : 00000001
> Processor Enabled : 1
> Performance Interrupt Trigger Mode : 0
> Virtual GIC Interrupt Trigger Mode : 0
> [268h 0616 4] Parking Protocol Version : 00000000
> [26Ch 0620 4] Performance Interrupt : 00000017
> [270h 0624 8] Parked Address : 0000000000000000
> [278h 0632 8] Base Address : 0000000008010000
> [280h 0640 8] Virtual GIC Base Address : 0000000000000000
> [288h 0648 8] Hypervisor GIC Base Address : 0000000000000000
> [290h 0656 4] Virtual GIC Interrupt : 00000000
> [294h 0660 8] Redistributor Base Address : 0000000000000000
> [29Ch 0668 8] ARM MPIDR : 0000000000000007
> /**** ACPI subtable terminates early - may be older version (dump table) */
>
> [2A4h 0676 1] Subtable Type : 0D [Generic MSI Frame]
> [2A5h 0677 1] Length : 18
> [2A6h 0678 2] Reserved : 0000
> [2A8h 0680 4] MSI Frame ID : 00000000
> [2ACh 0684 8] Base Address : 0000000008020000
> [2B4h 0692 4] Flags (decoded below) : 00000001
> Select SPI : 1
> [2B8h 0696 2] SPI Count : 0040
> [2BAh 0698 2] SPI Base : 0050
>
> Raw Table Data: Length 700 (0x2BC)
>
> 0000: 41 50 49 43 BC 02 00 00 03 18 42 4F 43 48 53 20 // APIC......BOCHS
> 0010: 42 58 50 43 41 50 49 43 01 00 00 00 42 58 50 43 // BXPCAPIC....BXPC
> 0020: 01 00 00 00 00 00 00 00 00 00 00 00 0C 18 00 00 // ................
> 0030: 00 00 00 00 00 00 00 08 00 00 00 00 00 00 00 00 // ................
> 0040: 02 00 00 00 0B 4C 00 00 00 00 00 00 00 00 00 00 // .....L..........
> 0050: 01 00 00 00 00 00 00 00 17 00 00 00 00 00 00 00 // ................
> 0060: 00 00 00 00 00 00 01 08 00 00 00 00 00 00 00 00 // ................
> 0070: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0090: 0B 4C 00 00 01 00 00 00 01 00 00 00 01 00 00 00 // .L..............
> 00A0: 00 00 00 00 17 00 00 00 00 00 00 00 00 00 00 00 // ................
> 00B0: 00 00 01 08 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 00C0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 00D0: 00 00 00 00 01 00 00 00 00 00 00 00 0B 4C 00 00 // .............L..
> 00E0: 02 00 00 00 02 00 00 00 01 00 00 00 00 00 00 00 // ................
> 00F0: 17 00 00 00 00 00 00 00 00 00 00 00 00 00 01 08 // ................
> 0100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0120: 02 00 00 00 00 00 00 00 0B 4C 00 00 03 00 00 00 // .........L......
> 0130: 03 00 00 00 01 00 00 00 00 00 00 00 17 00 00 00 // ................
> 0140: 00 00 00 00 00 00 00 00 00 00 01 08 00 00 00 00 // ................
> 0150: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0160: 00 00 00 00 00 00 00 00 00 00 00 00 03 00 00 00 // ................
> 0170: 00 00 00 00 0B 4C 00 00 04 00 00 00 04 00 00 00 // .....L..........
> 0180: 01 00 00 00 00 00 00 00 17 00 00 00 00 00 00 00 // ................
> 0190: 00 00 00 00 00 00 01 08 00 00 00 00 00 00 00 00 // ................
> 01A0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 01B0: 00 00 00 00 00 00 00 00 04 00 00 00 00 00 00 00 // ................
> 01C0: 0B 4C 00 00 05 00 00 00 05 00 00 00 01 00 00 00 // .L..............
> 01D0: 00 00 00 00 17 00 00 00 00 00 00 00 00 00 00 00 // ................
> 01E0: 00 00 01 08 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 01F0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0200: 00 00 00 00 05 00 00 00 00 00 00 00 0B 4C 00 00 // .............L..
> 0210: 06 00 00 00 06 00 00 00 01 00 00 00 00 00 00 00 // ................
> 0220: 17 00 00 00 00 00 00 00 00 00 00 00 00 00 01 08 // ................
> 0230: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0240: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0250: 06 00 00 00 00 00 00 00 0B 4C 00 00 07 00 00 00 // .........L......
> 0260: 07 00 00 00 01 00 00 00 00 00 00 00 17 00 00 00 // ................
> 0270: 00 00 00 00 00 00 00 00 00 00 01 08 00 00 00 00 // ................
> 0280: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 // ................
> 0290: 00 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 // ................
> 02A0: 00 00 00 00 0D 18 00 00 00 00 00 00 00 00 02 08 // ................
> 02B0: 00 00 00 00 01 00 00 00 40 00 50 00 // ........@.P.
I'm unsure if this table is supposed to lead to the creation of the
(apparently missing)
CPU#0 <-> numa-node#0
mapping, via
smp_init_cpus() [arch/arm64/kernel/smp.c]
acpi_table_parse_madt() [drivers/acpi/tables.c]
acpi_parse_gic_cpu_interface() [arch/arm64/kernel/smp.c]
acpi_map_gic_cpu_interface() [arch/arm64/kernel/smp.c]
early_map_cpu_to_node() [arch/arm64/mm/numa.c]
(7) I also tried "numa=off" in addition to "acpi=force", just in case;
it didn't make a difference.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-13 22:50 aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0") Laszlo Ersek
@ 2016-10-14 8:05 ` Andrew Jones
2016-10-14 13:18 ` Laszlo Ersek
0 siblings, 1 reply; 11+ messages in thread
From: Andrew Jones @ 2016-10-14 8:05 UTC (permalink / raw)
To: Laszlo Ersek
Cc: Zhen Lei, Will Deacon, main kernel list, linux-arm-kernel,
Ard Biesheuvel, Shannon Zhao, Wei Huang
On Fri, Oct 14, 2016 at 12:50:29AM +0200, Laszlo Ersek wrote:
> (4) Analysis (well, a lame attempt at that, because I have zero
> familiarity with this code). Let me quote the patch:
>
> > commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
> > Author: Zhen Lei <thunder.leizhen@huawei.com>
> > Date: Thu Sep 1 14:55:04 2016 +0800
> >
> > arm64/numa: remove the limitation that cpu0 must bind to node0
> >
> > 1. Remove the old binding code.
> > 2. Read the nid of cpu0 from dts.
> > 3. Fallback the nid of cpu0 to 0 when numa=off is set in bootargs.
> >
> > Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> > Signed-off-by: Will Deacon <will.deacon@arm.com>
> >
> > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> > index c3c08368a685..8b048e6ec34a 100644
> > --- a/arch/arm64/kernel/smp.c
> > +++ b/arch/arm64/kernel/smp.c
> > @@ -624,6 +624,7 @@ static void __init of_parse_and_init_cpus(void)
> > }
> >
> > bootcpu_valid = true;
> > + early_map_cpu_to_node(0, of_node_to_nid(dn));
> >
> > /*
> > * cpu_logical_map has already been
> > diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> > index 0a15f010b64a..778a985c8a70 100644
> > --- a/arch/arm64/mm/numa.c
> > +++ b/arch/arm64/mm/numa.c
> > @@ -116,16 +116,24 @@ static void __init setup_node_to_cpumask_map(void)
> > */
> > void numa_store_cpu_info(unsigned int cpu)
> > {
> > - map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
> > + map_cpu_to_node(cpu, cpu_to_node_map[cpu]);
> > }
> >
> > void __init early_map_cpu_to_node(unsigned int cpu, int nid)
> > {
> > /* fallback to node 0 */
> > - if (nid < 0 || nid >= MAX_NUMNODES)
> > + if (nid < 0 || nid >= MAX_NUMNODES || numa_off)
> > nid = 0;
The ACPI equivalent code must be missing (at least) the above, because,
even with DT, mach-virt won't have cpu to node mappings unless numa
is configured on the command line. Can you try adding something like
-m 512 -smp 4 \
-numa node,mem=256M,cpus=0-1,nodeid=0 \
-numa node,mem=256M,cpus=2-3,nodeid=1
to your QEMU command line? Then when you boot with ACPI you'll get a
SRAT. If that works, then we're just missing the "no SRAT, nid = 0"
code (that should have been added with this patch)
Thanks,
drew
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 8:05 ` Andrew Jones
@ 2016-10-14 13:18 ` Laszlo Ersek
2016-10-14 13:44 ` Andrew Jones
2016-10-14 15:01 ` Laszlo Ersek
0 siblings, 2 replies; 11+ messages in thread
From: Laszlo Ersek @ 2016-10-14 13:18 UTC (permalink / raw)
To: Andrew Jones
Cc: Zhen Lei, Will Deacon, main kernel list, linux-arm-kernel,
Ard Biesheuvel, Shannon Zhao, Wei Huang
On 10/14/16 10:05, Andrew Jones wrote:
> On Fri, Oct 14, 2016 at 12:50:29AM +0200, Laszlo Ersek wrote:
>> (4) Analysis (well, a lame attempt at that, because I have zero
>> familiarity with this code). Let me quote the patch:
>>
>>> commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
>>> Author: Zhen Lei <thunder.leizhen@huawei.com>
>>> Date: Thu Sep 1 14:55:04 2016 +0800
>>>
>>> arm64/numa: remove the limitation that cpu0 must bind to node0
>>>
>>> 1. Remove the old binding code.
>>> 2. Read the nid of cpu0 from dts.
>>> 3. Fallback the nid of cpu0 to 0 when numa=off is set in bootargs.
>>>
>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>>
>>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>>> index c3c08368a685..8b048e6ec34a 100644
>>> --- a/arch/arm64/kernel/smp.c
>>> +++ b/arch/arm64/kernel/smp.c
>>> @@ -624,6 +624,7 @@ static void __init of_parse_and_init_cpus(void)
>>> }
>>>
>>> bootcpu_valid = true;
>>> + early_map_cpu_to_node(0, of_node_to_nid(dn));
>>>
>>> /*
>>> * cpu_logical_map has already been
>>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>>> index 0a15f010b64a..778a985c8a70 100644
>>> --- a/arch/arm64/mm/numa.c
>>> +++ b/arch/arm64/mm/numa.c
>>> @@ -116,16 +116,24 @@ static void __init setup_node_to_cpumask_map(void)
>>> */
>>> void numa_store_cpu_info(unsigned int cpu)
>>> {
>>> - map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
>>> + map_cpu_to_node(cpu, cpu_to_node_map[cpu]);
>>> }
>>>
>>> void __init early_map_cpu_to_node(unsigned int cpu, int nid)
>>> {
>>> /* fallback to node 0 */
>>> - if (nid < 0 || nid >= MAX_NUMNODES)
>>> + if (nid < 0 || nid >= MAX_NUMNODES || numa_off)
>>> nid = 0;
>
> The ACPI equivalent code must be missing (at least) the above, because,
> even with DT, mach-virt won't have cpu to node mappings unless numa
> is configured on the command line. Can you try adding something like
>
> -m 512 -smp 4 \
> -numa node,mem=256M,cpus=0-1,nodeid=0 \
> -numa node,mem=256M,cpus=2-3,nodeid=1
>
> to your QEMU command line?
I added the following to my domain XML, under <cpu>:
<numa>
<cell id='0' cpus='0-1' memory='2097152' unit='KiB'/>
<cell id='1' cpus='2-3' memory='2097152' unit='KiB'/>
</numa>
(See <http://libvirt.org/formatdomain.html#elementsCPU>.)
With that, each NUMA node gets half of the VCPUs and half of the guest RAM.
(This is in a different guest now, one that has a bleeding edge Fedora kernel -- I didn't want to rebuild the upstream kernel yet again, just for this test. So, "4.9.0-0.rc0.git7.1.fc26.aarch64" is based on upstream v4.8-14109-g1573d2c, and it reproduces the problem too.)
> Then when you boot with ACPI you'll get a
> SRAT.
Yes, that's confirmed by the guest kernel log (see below).
> If that works, then we're just missing the "no SRAT, nid = 0"
> code (that should have been added with this patch)
It still crashes with the SRAT, with the following log:
> EFI stub: Booting Linux Kernel...
> ConvertPages: Incompatible memory types
> EFI stub: Using DTB from configuration table
> EFI stub: Exiting boot services and installing virtual address map...
> [ 0.000000] Booting Linux on physical CPU 0x0
> [ 0.000000] Linux version 4.9.0-0.rc0.git7.1.fc26.aarch64 (mockbuild@buildvm-aarch64-01.arm.fedoraproject.org) (gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC) ) #1 SMP Wed Oct 12 17:44:54 UTC 2016
> [ 0.000000] Boot CPU: AArch64 Processor [500f0000]
> [ 0.000000] efi: Getting EFI parameters from FDT:
> [ 0.000000] efi: EFI v2.60 by EDK II
> [ 0.000000] efi: SMBIOS 3.0=0xbbdb0000 ACPI 2.0=0xb86d0000 MEMATTR=0xb936b018
> [ 0.000000] cma: Reserved 512 MiB at 0x00000000e0000000
> [ 0.000000] ACPI: Early table checksum verification disabled
> [ 0.000000] ACPI: RSDP 0x00000000B86D0000 000024 (v02 BOCHS )
> [ 0.000000] ACPI: XSDT 0x00000000B86C0000 000054 (v01 BOCHS BXPCFACP 00000001 01000013)
> [ 0.000000] ACPI: FACP 0x00000000B83E0000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001)
> [ 0.000000] ACPI: DSDT 0x00000000B83F0000 0010E5 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: APIC 0x00000000B83D0000 00018C (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001)
> [ 0.000000] ACPI: GTDT 0x00000000B83C0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: MCFG 0x00000000B83B0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001)
> [ 0.000000] ACPI: SPCR 0x00000000B83A0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001)
> [ 0.000000] ACPI: SRAT 0x00000000B8390000 0000C8 (v03 BOCHS BXPCSRAT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: SPCR: console: pl011,mmio,0x9000000,9600
> [ 0.000000] earlycon: pl11 at MMIO 0x0000000009000000 (options '9600')
> [ 0.000000] bootconsole [pl11] enabled
> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0
> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0
> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x2 -> Node 1
> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x3 -> Node 1
> [ 0.000000] NUMA: Adding memblock [0x40000000 - 0xbfffffff] on node 0
> [ 0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x40000000-0xbfffffff]
> [ 0.000000] NUMA: Adding memblock [0xc0000000 - 0x13fffffff] on node 1
> [ 0.000000] ACPI: SRAT: Node 1 PXM 1 [mem 0xc0000000-0x13fffffff]
> [ 0.000000] NUMA: Initmem setup node 0 [mem 0x40000000-0xbfffffff]
> [ 0.000000] NUMA: NODE_DATA [mem 0xbfff2580-0xbfffffff]
> [ 0.000000] NUMA: Initmem setup node 1 [mem 0xc0000000-0x13fffffff]
> [ 0.000000] NUMA: NODE_DATA [mem 0x13fff2580-0x13fffffff]
> [ 0.000000] Zone ranges:
> [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]
> [ 0.000000] Normal [mem 0x0000000100000000-0x000000013fffffff]
> [ 0.000000] Movable zone start for each node
> [ 0.000000] Early memory node ranges
> [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000b838ffff]
> [ 0.000000] node 0: [mem 0x00000000b8390000-0x00000000b83fffff]
> [ 0.000000] node 0: [mem 0x00000000b8400000-0x00000000b841ffff]
> [ 0.000000] node 0: [mem 0x00000000b8420000-0x00000000b874ffff]
> [ 0.000000] node 0: [mem 0x00000000b8750000-0x00000000bbc1ffff]
> [ 0.000000] node 0: [mem 0x00000000bbc20000-0x00000000bbffffff]
> [ 0.000000] node 0: [mem 0x00000000bc000000-0x00000000bfffffff]
> [ 0.000000] node 1: [mem 0x00000000c0000000-0x000000013fffffff]
> [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000bfffffff]
> [ 0.000000] Initmem setup node 1 [mem 0x00000000c0000000-0x000000013fffffff]
> [ 0.000000] psci: probing for conduit method from ACPI.
> [ 0.000000] psci: PSCIv0.2 detected in firmware.
> [ 0.000000] psci: Using standard PSCI v0.2 function IDs
> [ 0.000000] psci: Trusted OS migration not required
> [ 0.000000] percpu: Embedded 3 pages/cpu @fffffe007fda0000 s117832 r8192 d70584 u196608
> [ 0.000000] Detected PIPT I-cache on CPU0
> [ 0.000000] Built 2 zonelists in Node order, mobility grouping on. Total pages: 65472
> [ 0.000000] Policy zone: Normal
> [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.9.0-0.rc0.git7.1.fc26.aarch64 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap LANG=en_US.UTF-8 earlycon acpi=force
> [ 0.000000] PID hash table entries: 4096 (order: -1, 32768 bytes)
> [ 0.000000] software IO TLB [mem 0xdbff0000-0xdfff0000] (64MB) mapped at [fffffe009bff0000-fffffe009ffeffff]
> [ 0.000000] Memory: 3542976K/4194304K available (9148K kernel code, 1612K rwdata, 3776K rodata, 1600K init, 15899K bss, 127040K reserved, 524288K cma-reserved)
> [ 0.000000] Virtual kernel memory layout:
> [ 0.000000] modules : 0xfffffc0000000000 - 0xfffffc0008000000 ( 128 MB)
> vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000 ( 2045 GB)
> .text : 0xfffffc0008080000 - 0xfffffc0008970000 ( 9152 KB)
> .rodata : 0xfffffc0008970000 - 0xfffffc0008d30000 ( 3840 KB)
> .init : 0xfffffc0008d30000 - 0xfffffc0008ec0000 ( 1600 KB)
> .data : 0xfffffc0008ec0000 - 0xfffffc0009053200 ( 1613 KB)
> .bss : 0xfffffc0009053200 - 0xfffffc0009fda058 ( 15900 KB)
> fixed : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000 ( 4288 KB)
> PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000 ( 16 MB)
> vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000 ( 2 GB maximum)
> 0xfffffdff80000000 - 0xfffffdff80400000 ( 4 MB actual)
> memory : 0xfffffe0000000000 - 0xfffffe0100000000 ( 4096 MB)
> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=2
> [ 0.000000] Running RCU self tests
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] RCU lockdep checking is enabled.
> [ 0.000000] Build-time adjustment of leaf fanout to 64.
> [ 0.000000] RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
> [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
> [ 0.000000] kmemleak: Kernel memory leak detector disabled
> [ 0.000000] NR_IRQS:64 nr_irqs:64 0
> [ 0.000000] GICv2m: ACPI overriding V2M MSI_TYPER (base:80, num:64)
> [ 0.000000] GICv2m: range[mem 0x08020000-0x08020fff], SPI[80:143]
> [ 0.000000] GIC: PPI11 is secure or misconfigured
> [ 0.000000] arm_arch_timer: WARNING: Invalid trigger for IRQ3, assuming level low
> [ 0.000000] arm_arch_timer: WARNING: Please fix your firmware
> [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 50.00MHz (virt).
> [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
> [ 0.000003] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
> [ 0.002198] Console: colour dummy device 80x25
> [ 0.003319] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> [ 0.005236] ... MAX_LOCKDEP_SUBCLASSES: 8
> [ 0.006183] ... MAX_LOCK_DEPTH: 48
> [ 0.007273] ... MAX_LOCKDEP_KEYS: 8191
> [ 0.008287] ... CLASSHASH_SIZE: 4096
> [ 0.009296] ... MAX_LOCKDEP_ENTRIES: 32768
> [ 0.010327] ... MAX_LOCKDEP_CHAINS: 65536
> [ 0.011318] ... CHAINHASH_SIZE: 32768
> [ 0.012453] memory used by lock dependency info: 8159 kB
> [ 0.013736] per task-struct memory footprint: 1920 bytes
> [ 0.015742] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
> [ 0.018710] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=50000)
> [ 0.021221] pid_max: default: 32768 minimum: 301
> [ 0.022806] ACPI: Core revision 20160831
> [ 0.027885] ACPI: 1 ACPI AML tables successfully acquired and loaded
>
> [ 0.030252] Security Framework initialized
> [ 0.031355] Yama: becoming mindful.
> [ 0.032176] SELinux: Initializing.
> [ 0.033925] Dentry cache hash table entries: 524288 (order: 6, 4194304 bytes)
> [ 0.037039] Inode-cache hash table entries: 262144 (order: 5, 2097152 bytes)
> [ 0.039383] Mount-cache hash table entries: 8192 (order: 0, 65536 bytes)
> [ 0.041135] Mountpoint-cache hash table entries: 8192 (order: 0, 65536 bytes)
> [ 0.044725] ftrace: allocating 29596 entries in 8 pages
> [ 0.080467] ASID allocator initialised with 65536 entries
> [ 0.082070] ------------[ cut here ]------------
> [ 0.083227] WARNING: CPU: 0 PID: 1 at kernel/workqueue.c:5458 wq_numa_init+0x178/0x21c
> [ 0.085304] Modules linked in:
> [ 0.086102]
> [ 0.086499] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.9.0-0.rc0.git7.1.fc26.aarch64 #1
> [ 0.088611] Hardware name: linux,dummy-virt (DT)
> [ 0.089816] task: fffffe00700aac00 task.stack: fffffe00f8044000
> [ 0.091375] PC is at wq_numa_init+0x178/0x21c
> [ 0.092514] LR is at wq_numa_init+0x14c/0x21c
> [ 0.093654] pc : [<fffffc0008d3f434>] lr : [<fffffc0008d3f408>] pstate: 60000045
> [ 0.095589] sp : fffffe00f8047cb0
> [ 0.096457] x29: fffffe00f8047cb0 [ 0.097311] x28: 0000000000000000
> [ 0.098201]
> [ 0.098601] x27: 0000000000000000 [ 0.099450] x26: fffffc0008ef4a28
> [ 0.100342]
> [ 0.100730] x25: fffffc0008ef3000 [ 0.101576] x24: fffffc0008ef3574
> [ 0.102466]
> [ 0.102853] x23: 0000000000000000 [ 0.103700] x22: fffffe007937de00
> [ 0.104593]
> [ 0.104982] x21: fffffc0008e887f8 [ 0.105829] x20: fffffc0009091000
> [ 0.106723]
> [ 0.107111] x19: 0000000000000000 [ 0.107956] x18: 0000000050642c6a
> [ 0.108847]
> [ 0.109234] x17: 0000000000000000 [ 0.110078] x16: 0000000000000000
> [ 0.110968]
> [ 0.111363] x15: 00000000fcacdc89 [ 0.112199] x14: 0000000000000000
> [ 0.113087]
> [ 0.113481] x13: 0000000000000000 [ 0.114324] x12: 00000000fe2ce6e0
> [ 0.115204]
> [ 0.115597] x11: 0000000000000001 [ 0.116439] x10: 0000000000000048
> [ 0.117328]
> [ 0.117716] x9 : 0000000000000000 [ 0.118563] x8 : fffffe00f4010080
> [ 0.119453]
> [ 0.119833] x7 : 0000000000000000 [ 0.120678] x6 : 0000000000000000
> [ 0.121571]
> [ 0.121959] x5 : 000000000000000f [ 0.122804] x4 : 0000000000000000
> [ 0.123695]
> [ 0.124084] x3 : 0000000000000000 [ 0.124922] x2 : 0000000000000000
> [ 0.125815]
> [ 0.126204] x1 : 0000000000000004 [ 0.127055] x0 : 00000000ffffffff
> [ 0.127966]
> [ 0.128361]
> [ 0.128767] ---[ end trace 0000000000000000 ]---
> [ 0.129983] Call trace:
> [ 0.130629] Exception stack(0xfffffe00f8047ad0 to 0xfffffe00f8047c00)
> [ 0.132316] 7ac0: 0000000000000000 0000040000000000
> [ 0.134360] 7ae0: fffffe00f8047cb0 fffffc0008d3f434 0000000060000045 000000000000003d
> [ 0.136405] 7b00: fffffc0008ef4000 fffffe007937df00 0000000000000000 0000000000000000
> [ 0.138446] 7b20: fffffc0008bf4110 0000000000000189 0000000000000018 0000000000000028
> [ 0.140498] 7b40: fffffe00f8047b80 0000000000000000 fffffe0000000000 fffffc000848af30
> [ 0.142541] 7b60: fffffe00f8047ba0 fffffc0008134d24 fffffe00f8044000 0000000000000040
> [ 0.144558] 7b80: 00000000ffffffff 0000000000000004 0000000000000000 0000000000000000
> [ 0.146607] 7ba0: 0000000000000000 000000000000000f 0000000000000000 0000000000000000
> [ 0.148664] 7bc0: fffffe00f4010080 0000000000000000 0000000000000048 0000000000000001
> [ 0.150704] 7be0: 00000000fe2ce6e0 0000000000000000 0000000000000000 00000000fcacdc89
> [ 0.152752] [<fffffc0008d3f434>] wq_numa_init+0x178/0x21c
> [ 0.154160] [<fffffc0008d3f578>] init_workqueues+0xa0/0x4b8
> [ 0.155596] [<fffffc0008083594>] do_one_initcall+0x44/0x138
> [ 0.157059] [<fffffc0008d30d28>] kernel_init_freeable+0x178/0x2dc
> [ 0.158670] [<fffffc0008956f48>] kernel_init+0x18/0x110
> [ 0.160036] [<fffffc0008083330>] ret_from_fork+0x10/0x20
> [ 0.161440] workqueue: NUMA node mapping not available for cpu0, disabling NUMA support
> [ 0.165296] Remapping and enabling EFI services.
> [ 0.166586] Unable to handle kernel paging request at virtual address b91000006be8
> [ 0.168448] pgd = fffffc000a010000
> [ 0.169341] [b91000006be8] *pgd=0000000000000000[ 0.170505] , *pud=0000000000000000
> , *pmd=0000000000000000[ 0.171942]
> [ 0.172332] Internal error: Oops: 96000004 [#1] SMP
> [ 0.173600] Modules linked in:
> [ 0.174407] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.9.0-0.rc0.git7.1.fc26.aarch64 #1
> [ 0.176836] Hardware name: linux,dummy-virt (DT)
> [ 0.178038] task: fffffe00700aac00 task.stack: fffffe00f8044000
> [ 0.179579] PC is at __ll_sc_atomic_add+0x20/0x40
> [ 0.180800] LR is at __lock_acquire+0xe8/0x698
> [ 0.181961] pc : [<fffffc0008487390>] lr : [<fffffc0008138c08>] pstate: 800000c5
> [ 0.183895] sp : fffffe00f8047820
> [ 0.184755] x29: fffffe00f8047820 [ 0.185588] x28: fffffc0008ef3000
> [ 0.186479]
> [ 0.186868] x27: fffffc0008ef2358 [ 0.187713] x26: fffffc0009ce6000
> [ 0.188606]
> [ 0.188997] x25: 0000000000000001 [ 0.189857] x24: 0000000000000000
> [ 0.190731]
> [ 0.191115] x23: fffffe00700aac00 [ 0.191951] x22: 0000000000000000
> [ 0.192843]
> [ 0.193231] x21: fffffe007fd9a018 [ 0.194074] x20: 0000000000000000
> [ 0.194966]
> [ 0.195361] x19: fffffe007fd9a018 [ 0.196192] x18: 0000000000000010
> [ 0.197077]
> [ 0.197476] x17: 0000000057181979 [ 0.198325] x16: 0000000000000000
> [ 0.199209]
> [ 0.199604] x15: 0000000000000000 [ 0.200450] x14: 0000000000000000
> [ 0.201337]
> [ 0.201723] x13: 0000000000000001 [ 0.202555] x12: fffffe007fff2580
> [ 0.203432]
> [ 0.203819] x11: 0000000000000000 [ 0.204664] x10: 0000000000000011
> [ 0.205550]
> [ 0.205937] x9 : 0000000000000001 [ 0.206784] x8 : 0000b91000006be8
> [ 0.207678]
> [ 0.208062] x7 : fffffc0008299fcc [ 0.208899] x6 : 0000000000000000
> [ 0.209787]
> [ 0.210176] x5 : 0000000000000080 [ 0.211022] x4 : 0000b91000006a50
> [ 0.211913]
> [ 0.212307] x3 : 0000000000000000 [ 0.213147] x2 : 000022c80000f420
> [ 0.214034]
> [ 0.214421] x1 : 0000b91000006be8 [ 0.215251] x0 : fffffc0008138c08
> [ 0.216134]
> [ 0.216527]
> [ 0.216916] Process swapper/0 (pid: 1, stack limit = 0xfffffe00f8044020)
> [ 0.218671] Stack: (0xfffffe00f8047820 to 0xfffffe00f8048000)
> [ 0.220167] 7820: fffffe00f8047840 fffffc0008138c08 fffffe00f8044000 0000000000000001
> [ 0.222190] 7840: fffffe00f80478c0 fffffc0008139590 fffffe007fd9a018 0000000000000000
> [ 0.224238] 7860: 0000000000000000 0000000000000000 0000000000000001 0000000000000000
> [ 0.226284] 7880: fffffc0008299fcc 00000000000000c0 fffffc0008ef2358 fffffc0008ef3000
> [ 0.228318] 78a0: 0000000000000001 fffffc0009ce6000 0000000000000000 fffffe0000000000
> [ 0.230362] 78c0: fffffe00f8047930 fffffc000895f2c4 fffffe007fd9a000 fffffc0008299fcc
> [ 0.232394] 78e0: fffffe007fd9a000 fffffc000829ad94 fffffe007001db00 000000000000e8e8
> [ 0.234435] 7900: fffffe007001db00 fffffe007001dbf8 fffffe00fff3ef50 0000000000000000
> [ 0.236481] 7920: fffffe00f8047a20 fffffc0008ef2000 fffffe00f8047950 fffffc0008299fcc
> [ 0.238516] 7940: 00000000ffffffff fffffe007fd9a000 fffffe00f8047a70 fffffc000829aa68
> [ 0.240560] 7960: 00000000ffffffff 0000000000000001 00000000024000c0 fffffc000829ad94
> [ 0.242604] 7980: 0000000000210d00 000000000000e8e8 fffffe007001db00 fffffe007001dbf8
> [ 0.244634] 79a0: fffffe00fff3ef50 0000000000000000 fffffe00f8044000 0000000000000040
> [ 0.246678] 79c0: fffffc000828d620 fffffc0008ef3000 00000000026080c0 fffffe00fff3ef60
> [ 0.248733] 79e0: fffffe00f8047a00 fffffc00024000c0 fffffc0008f89000 0000000000000000
> [ 0.250783] 7a00: fffffe00f8047a20 fffffc000822f62c fffffc0009016b30 fffffe00f8047b40
> [ 0.252896] 7a20: fffffe00f8047ba0 fffffc000828d620 0000000000000000 fffffc0008ef0b28
> [ 0.255009] 7a40: fffffe007fff3c00 0000000000000000 0000000000000000 0000000000000000
> [ 0.257121] 7a60: fffffe00f8044000 0000000000000000 fffffe00f8047b90 fffffc000829ad94
> [ 0.259240] 7a80: 0000000000000040 fffffe007001db00 00000000024000c0 00000000ffffffff
> [ 0.261358] 7aa0: fffffc0008266284 fffffe00fff3ef50 0000000020000000 00e8000000000f07
> [ 0.263472] 7ac0: 0000000000000000 0000000000000400 fffffc0008f89000 0000000000000000
> [ 0.265662] 7ae0: fffffe00f8047b00 fffffc000822f62c fffffe00fff3ef60 0000000000000000
> [ 0.267787] 7b00: 0000001000000000 fffffc0008266284 fffffe00f8047b50 fffffc0008134d24
> [ 0.269905] 7b20: fffffe00f8044000 0000000000000040 fffffc0008bf4110 0000000000000189
> [ 0.272020] 7b40: fffffc0008ef4000 0000000000000000 fffffe00f8047b70 fffffc000810267c
> [ 0.274136] 7b60: fffffc0009016893 0000000000000000 fffffe00f8047ba0 fffffc0008102784
> [ 0.276250] 7b80: fffffe00f8047b90 fffffc000829ad7c fffffe00f8047bd0 fffffc000829b13c
> [ 0.278371] 7ba0: fffffe007001db00 00000000024000c0 fffffc0008266284 fffffe007001db00
> [ 0.280484] 7bc0: fffffc0008ef4000 0000000000000000 fffffe00f8047c30 fffffc0008266284
> [ 0.282600] 7be0: fffffdff801b0200 fffffe006c080000 000000006c080000 0000000020000000
> [ 0.284715] 7c00: fffffe00f0010008 0000000004000000 0000000020000000 00e8000000000f07
> [ 0.286831] 7c20: 0000000000000000 0000000000000000 fffffe00f8047c50 fffffc0008098e24
> [ 0.288948] 7c40: fffffdff801b0200 0000000000000001 fffffe00f8047c80 fffffc00080991d0
> [ 0.291062] 7c60: 0000000024000000 0000000000000001 0000000024000000 fffffc0008ef0b28
> [ 0.293178] 7c80: fffffe00f8047d00 fffffc0008d361cc fffffe0078416018 00e8000000000707
> [ 0.295296] 7ca0: fffffc0008ff6410 fffffc0008ef7000 0000000000000000 fffffc0008ff6410
> [ 0.297408] 7cc0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.299523] 7ce0: 0000000000000000 00e8000000000f05 fffffc0008098dd0 0000000023ffffff
> [ 0.301636] 7d00: fffffe00f8047d10 fffffc0008d35020 fffffe00f8047d40 fffffc0008d88284
> [ 0.303748] 7d20: fffffe0078416018 fffffc0008ff6000 fffffc0008c87348 fffffc0008d8821c
> [ 0.305863] 7d40: fffffe00f8047d90 fffffc0008083594 fffffc0008d88154 fffffe00f8044000
> [ 0.307987] 7d60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.310099] 7d80: 0000000000000000 0000000004000000 fffffe00f8047e00 fffffc0008d30d28
> [ 0.312217] 7da0: fffffc0008e622d8 fffffc0008e622e0 0000000000000040 0000000000000000
> [ 0.314333] 7dc0: fffffe00f8047e00 fffffc0008d30d18 fffffc0008e62220 fffffc0008e622e0
> [ 0.316445] 7de0: 0000000000000040 0000000000000000 0000000000000000 fffffc0008e622e0
> [ 0.318572] 7e00: fffffe00f8047ea0 fffffc0008956f48 fffffc0008956f30 0000000000000000
> [ 0.320692] 7e20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.322805] 7e40: 0000000000000000 0000000000000000 0000000000000000 0000000000000001
> [ 0.324914] 7e60: 0000000000000003 0000000000000000 0000000000000000 0000000000000000
> [ 0.327027] 7e80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.329139] 7ea0: 0000000000000000 fffffc0008083330 fffffc0008956f30 0000000000000000
> [ 0.331248] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.333361] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.335470] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.337585] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.339695] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.341810] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.343923] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.346037] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.348154] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
> [ 0.350272] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> [ 0.352392] Call trace:
> [ 0.353049] Exception stack(0xfffffe00f8047650 to 0xfffffe00f8047780)
> [ 0.354792] 7640: fffffe007fd9a018 0000040000000000
> [ 0.356910] 7660: fffffe00f8047820 fffffc0008487390 fffffe00f80476e0 fffffc0008131290
> [ 0.359025] 7680: fffffc000901690b fffffc0008f1e000 0000000000000001 fffffe00700aac00
> [ 0.361140] 76a0: fffffc000901690b fffffc0008f27a28 fffffe00fff3b700 fffffc0008e8b700
> [ 0.363255] 76c0: fffffe00fff3b700 fffffc0008ef1000 fffffe00f80476e0 00000000000000c0
> [ 0.365373] 76e0: fffffe00f8047720 fffffc000811a374 fffffc0008138c08 0000b91000006be8
> [ 0.367483] 7700: 000022c80000f420 0000000000000000 0000b91000006a50 0000000000000080
> [ 0.369593] 7720: 0000000000000000 fffffc0008299fcc 0000b91000006be8 0000000000000001
> [ 0.371702] 7740: 0000000000000011 0000000000000000 fffffe007fff2580 0000000000000001
> [ 0.373817] 7760: 0000000000000000 0000000000000000 0000000000000000 0000000057181979
> [ 0.375935] [<fffffc0008487390>] __ll_sc_atomic_add+0x20/0x40
> [ 0.377489] [<fffffc0008138c08>] __lock_acquire+0xe8/0x698
> [ 0.378960] [<fffffc0008139590>] lock_acquire+0xd8/0x2c0
> [ 0.380394] [<fffffc000895f2c4>] _raw_spin_lock+0x4c/0x60
> [ 0.381843] [<fffffc0008299fcc>] get_partial_node.isra.23+0x4c/0x440
> [ 0.383559] [<fffffc000829aa68>] ___slab_alloc+0x438/0x710
> [ 0.385031] [<fffffc000829ad94>] __slab_alloc+0x54/0xa0
> [ 0.386441] [<fffffc000829b13c>] kmem_cache_alloc+0x35c/0x428
> [ 0.387983] [<fffffc0008266284>] ptlock_alloc+0x2c/0x58
> [ 0.389394] [<fffffc0008098e24>] pgd_pgtable_alloc+0x54/0xd8
> [ 0.390912] [<fffffc00080991d0>] __create_pgd_mapping+0x158/0x2a8
> [ 0.392556] [<fffffc0008d361cc>] create_pgd_mapping+0x30/0x38
> [ 0.394100] [<fffffc0008d35020>] efi_create_mapping+0xfc/0x110
> [ 0.395682] [<fffffc0008d88284>] arm_enable_runtime_services+0x130/0x204
> [ 0.397501] [<fffffc0008083594>] do_one_initcall+0x44/0x138
> [ 0.399001] [<fffffc0008d30d28>] kernel_init_freeable+0x178/0x2dc
> [ 0.400646] [<fffffc0008956f48>] kernel_init+0x18/0x110
> [ 0.402053] [<fffffc0008083330>] ret_from_fork+0x10/0x20
> [ 0.403488] Code: aa1e03e0 aa0103e8 d503201f f9800111 (885f7d00)
> [ 0.405145] ---[ end trace f6be31446b0a9526 ]---
> [ 0.406286] note: swapper/0[1] exited with preempt_count 1
> [ 0.407687] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> [ 0.407687]
> [ 0.410047] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
> [ 0.410047]
>
This log contains two call traces. The first is a WARNING in wq_numa_init(). The second is the unhandled page fault.
Note the warning message (from wq_numa_init()):
workqueue: NUMA node mapping not available for cpu0, disabling NUMA support
Something looks genuinely broken with the cpu <-> numa-node associations in the ACPI case -- it even seems to fail when the SRAT does exist.
So, perhaps, commit 7ba5f605f3a0 may not have introduced the bug, only exposed one in the ACPI code?...
Thanks
Laszlo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 13:18 ` Laszlo Ersek
@ 2016-10-14 13:44 ` Andrew Jones
2016-10-14 15:45 ` Laszlo Ersek
2016-10-14 15:01 ` Laszlo Ersek
1 sibling, 1 reply; 11+ messages in thread
From: Andrew Jones @ 2016-10-14 13:44 UTC (permalink / raw)
To: Laszlo Ersek
Cc: Zhen Lei, Will Deacon, main kernel list, linux-arm-kernel,
Ard Biesheuvel, Shannon Zhao, Wei Huang
On Fri, Oct 14, 2016 at 03:18:00PM +0200, Laszlo Ersek wrote:
> On 10/14/16 10:05, Andrew Jones wrote:
> > On Fri, Oct 14, 2016 at 12:50:29AM +0200, Laszlo Ersek wrote:
> >> (4) Analysis (well, a lame attempt at that, because I have zero
> >> familiarity with this code). Let me quote the patch:
> >>
> >>> commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
> >>> Author: Zhen Lei <thunder.leizhen@huawei.com>
> >>> Date: Thu Sep 1 14:55:04 2016 +0800
> >>>
> >>> arm64/numa: remove the limitation that cpu0 must bind to node0
> >>>
> >>> 1. Remove the old binding code.
> >>> 2. Read the nid of cpu0 from dts.
> >>> 3. Fallback the nid of cpu0 to 0 when numa=off is set in bootargs.
> >>>
> >>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
> >>> Signed-off-by: Will Deacon <will.deacon@arm.com>
> >>>
> >>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> >>> index c3c08368a685..8b048e6ec34a 100644
> >>> --- a/arch/arm64/kernel/smp.c
> >>> +++ b/arch/arm64/kernel/smp.c
> >>> @@ -624,6 +624,7 @@ static void __init of_parse_and_init_cpus(void)
> >>> }
> >>>
> >>> bootcpu_valid = true;
> >>> + early_map_cpu_to_node(0, of_node_to_nid(dn));
> >>>
> >>> /*
> >>> * cpu_logical_map has already been
> >>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
> >>> index 0a15f010b64a..778a985c8a70 100644
> >>> --- a/arch/arm64/mm/numa.c
> >>> +++ b/arch/arm64/mm/numa.c
> >>> @@ -116,16 +116,24 @@ static void __init setup_node_to_cpumask_map(void)
> >>> */
> >>> void numa_store_cpu_info(unsigned int cpu)
> >>> {
> >>> - map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
> >>> + map_cpu_to_node(cpu, cpu_to_node_map[cpu]);
> >>> }
> >>>
> >>> void __init early_map_cpu_to_node(unsigned int cpu, int nid)
> >>> {
> >>> /* fallback to node 0 */
> >>> - if (nid < 0 || nid >= MAX_NUMNODES)
> >>> + if (nid < 0 || nid >= MAX_NUMNODES || numa_off)
> >>> nid = 0;
> >
> > The ACPI equivalent code must be missing (at least) the above, because,
> > even with DT, mach-virt won't have cpu to node mappings unless numa
> > is configured on the command line. Can you try adding something like
> >
> > -m 512 -smp 4 \
> > -numa node,mem=256M,cpus=0-1,nodeid=0 \
> > -numa node,mem=256M,cpus=2-3,nodeid=1
> >
> > to your QEMU command line?
>
> I added the following to my domain XML, under <cpu>:
>
> <numa>
> <cell id='0' cpus='0-1' memory='2097152' unit='KiB'/>
> <cell id='1' cpus='2-3' memory='2097152' unit='KiB'/>
> </numa>
>
> (See <http://libvirt.org/formatdomain.html#elementsCPU>.)
>
> With that, each NUMA node gets half of the VCPUs and half of the guest RAM.
>
> (This is in a different guest now, one that has a bleeding edge Fedora kernel -- I didn't want to rebuild the upstream kernel yet again, just for this test. So, "4.9.0-0.rc0.git7.1.fc26.aarch64" is based on upstream v4.8-14109-g1573d2c, and it reproduces the problem too.)
>
> > Then when you boot with ACPI you'll get a
> > SRAT.
>
> Yes, that's confirmed by the guest kernel log (see below).
>
> > If that works, then we're just missing the "no SRAT, nid = 0"
> > code (that should have been added with this patch)
>
> It still crashes with the SRAT, with the following log:
Rats.
> Note the warning message (from wq_numa_init()):
>
> workqueue: NUMA node mapping not available for cpu0, disabling NUMA support
>
> Something looks genuinely broken with the cpu <-> numa-node associations in the ACPI case -- it even seems to fail when the SRAT does exist.
>
> So, perhaps, commit 7ba5f605f3a0 may not have introduced the bug, only exposed one in the ACPI code?...
The kernel's ACPI NUMA support used to work. It was the test case for
QEMU's SRAT generation code.
Two more experiments I wouldn't mind having you try, if you have time;
1) confirm that this NUMA configured guest and this latest kernel works
with DT. This is just for sanity, but I guess it will.
2) If (1) succeeds, then try the last-known-good kernel with ACPI again,
but this time with the NUMA configured guest.
If (2) fails then we may need to expand the bisection :-(
Thanks,
drew
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 13:18 ` Laszlo Ersek
2016-10-14 13:44 ` Andrew Jones
@ 2016-10-14 15:01 ` Laszlo Ersek
2016-10-14 15:27 ` Laszlo Ersek
1 sibling, 1 reply; 11+ messages in thread
From: Laszlo Ersek @ 2016-10-14 15:01 UTC (permalink / raw)
To: Andrew Jones, Zhen Lei, Will Deacon, Lorenzo Pieralisi, Hanjun Guo
Cc: main kernel list, linux-arm-kernel, Ard Biesheuvel, Shannon Zhao,
Wei Huang
On 10/14/16 15:18, Laszlo Ersek wrote:
> On 10/14/16 10:05, Andrew Jones wrote:
>> On Fri, Oct 14, 2016 at 12:50:29AM +0200, Laszlo Ersek wrote:
>>> (4) Analysis (well, a lame attempt at that, because I have zero
>>> familiarity with this code). Let me quote the patch:
>>>
>>>> commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
>>>> Author: Zhen Lei <thunder.leizhen@huawei.com>
>>>> Date: Thu Sep 1 14:55:04 2016 +0800
>>>>
>>>> arm64/numa: remove the limitation that cpu0 must bind to node0
>>>>
>>>> 1. Remove the old binding code.
>>>> 2. Read the nid of cpu0 from dts.
>>>> 3. Fallback the nid of cpu0 to 0 when numa=off is set in bootargs.
>>>>
>>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
>>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>>>
>>>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>>>> index c3c08368a685..8b048e6ec34a 100644
>>>> --- a/arch/arm64/kernel/smp.c
>>>> +++ b/arch/arm64/kernel/smp.c
>>>> @@ -624,6 +624,7 @@ static void __init of_parse_and_init_cpus(void)
>>>> }
>>>>
>>>> bootcpu_valid = true;
>>>> + early_map_cpu_to_node(0, of_node_to_nid(dn));
>>>>
>>>> /*
>>>> * cpu_logical_map has already been
>>>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>>>> index 0a15f010b64a..778a985c8a70 100644
>>>> --- a/arch/arm64/mm/numa.c
>>>> +++ b/arch/arm64/mm/numa.c
>>>> @@ -116,16 +116,24 @@ static void __init setup_node_to_cpumask_map(void)
>>>> */
>>>> void numa_store_cpu_info(unsigned int cpu)
>>>> {
>>>> - map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
>>>> + map_cpu_to_node(cpu, cpu_to_node_map[cpu]);
>>>> }
>>>>
>>>> void __init early_map_cpu_to_node(unsigned int cpu, int nid)
>>>> {
>>>> /* fallback to node 0 */
>>>> - if (nid < 0 || nid >= MAX_NUMNODES)
>>>> + if (nid < 0 || nid >= MAX_NUMNODES || numa_off)
>>>> nid = 0;
>>
>> The ACPI equivalent code must be missing (at least) the above,
>> because, even with DT, mach-virt won't have cpu to node mappings
>> unless numa is configured on the command line. Can you try adding
>> something like
>>
>> -m 512 -smp 4 \
>> -numa node,mem=256M,cpus=0-1,nodeid=0 \
>> -numa node,mem=256M,cpus=2-3,nodeid=1
>>
>> to your QEMU command line?
>
> I added the following to my domain XML, under <cpu>:
>
> <numa>
> <cell id='0' cpus='0-1' memory='2097152' unit='KiB'/>
> <cell id='1' cpus='2-3' memory='2097152' unit='KiB'/>
> </numa>
>
> (See <http://libvirt.org/formatdomain.html#elementsCPU>.)
>
> With that, each NUMA node gets half of the VCPUs and half of the guest
> RAM.
>
> (This is in a different guest now, one that has a bleeding edge Fedora
> kernel -- I didn't want to rebuild the upstream kernel yet again, just
> for this test. So, "4.9.0-0.rc0.git7.1.fc26.aarch64" is based on
> upstream v4.8-14109-g1573d2c, and it reproduces the problem too.)
>
>> Then when you boot with ACPI you'll get a
>> SRAT.
>
> Yes, that's confirmed by the guest kernel log (see below).
>
>> If that works, then we're just missing the "no SRAT, nid = 0"
>> code (that should have been added with this patch)
>
> It still crashes with the SRAT, with the following log:
>
>> EFI stub: Booting Linux Kernel...
>> ConvertPages: Incompatible memory types
>> EFI stub: Using DTB from configuration table
>> EFI stub: Exiting boot services and installing virtual address map...
>> [ 0.000000] Booting Linux on physical CPU 0x0
>> [ 0.000000] Linux version 4.9.0-0.rc0.git7.1.fc26.aarch64 (mockbuild@buildvm-aarch64-01.arm.fedoraproject.org) (gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC) ) #1 SMP Wed Oct 12 17:44:54 UTC 2016
>> [ 0.000000] Boot CPU: AArch64 Processor [500f0000]
>> [ 0.000000] efi: Getting EFI parameters from FDT:
>> [ 0.000000] efi: EFI v2.60 by EDK II
>> [ 0.000000] efi: SMBIOS 3.0=0xbbdb0000 ACPI 2.0=0xb86d0000 MEMATTR=0xb936b018
>> [ 0.000000] cma: Reserved 512 MiB at 0x00000000e0000000
>> [ 0.000000] ACPI: Early table checksum verification disabled
>> [ 0.000000] ACPI: RSDP 0x00000000B86D0000 000024 (v02 BOCHS )
>> [ 0.000000] ACPI: XSDT 0x00000000B86C0000 000054 (v01 BOCHS BXPCFACP 00000001 01000013)
>> [ 0.000000] ACPI: FACP 0x00000000B83E0000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: DSDT 0x00000000B83F0000 0010E5 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: APIC 0x00000000B83D0000 00018C (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: GTDT 0x00000000B83C0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: MCFG 0x00000000B83B0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: SPCR 0x00000000B83A0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: SRAT 0x00000000B8390000 0000C8 (v03 BOCHS BXPCSRAT 00000001 BXPC 00000001)
>> [ 0.000000] ACPI: SPCR: console: pl011,mmio,0x9000000,9600
>> [ 0.000000] earlycon: pl11 at MMIO 0x0000000009000000 (options '9600')
>> [ 0.000000] bootconsole [pl11] enabled
>> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0
>> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0
>> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x2 -> Node 1
>> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x3 -> Node 1
>> [ 0.000000] NUMA: Adding memblock [0x40000000 - 0xbfffffff] on node 0
>> [ 0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x40000000-0xbfffffff]
>> [ 0.000000] NUMA: Adding memblock [0xc0000000 - 0x13fffffff] on node 1
>> [ 0.000000] ACPI: SRAT: Node 1 PXM 1 [mem 0xc0000000-0x13fffffff]
>> [ 0.000000] NUMA: Initmem setup node 0 [mem 0x40000000-0xbfffffff]
>> [ 0.000000] NUMA: NODE_DATA [mem 0xbfff2580-0xbfffffff]
>> [ 0.000000] NUMA: Initmem setup node 1 [mem 0xc0000000-0x13fffffff]
>> [ 0.000000] NUMA: NODE_DATA [mem 0x13fff2580-0x13fffffff]
>> [ 0.000000] Zone ranges:
>> [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]
>> [ 0.000000] Normal [mem 0x0000000100000000-0x000000013fffffff]
>> [ 0.000000] Movable zone start for each node
>> [ 0.000000] Early memory node ranges
>> [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000b838ffff]
>> [ 0.000000] node 0: [mem 0x00000000b8390000-0x00000000b83fffff]
>> [ 0.000000] node 0: [mem 0x00000000b8400000-0x00000000b841ffff]
>> [ 0.000000] node 0: [mem 0x00000000b8420000-0x00000000b874ffff]
>> [ 0.000000] node 0: [mem 0x00000000b8750000-0x00000000bbc1ffff]
>> [ 0.000000] node 0: [mem 0x00000000bbc20000-0x00000000bbffffff]
>> [ 0.000000] node 0: [mem 0x00000000bc000000-0x00000000bfffffff]
>> [ 0.000000] node 1: [mem 0x00000000c0000000-0x000000013fffffff]
>> [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x00000000bfffffff]
>> [ 0.000000] Initmem setup node 1 [mem 0x00000000c0000000-0x000000013fffffff]
>> [ 0.000000] psci: probing for conduit method from ACPI.
>> [ 0.000000] psci: PSCIv0.2 detected in firmware.
>> [ 0.000000] psci: Using standard PSCI v0.2 function IDs
>> [ 0.000000] psci: Trusted OS migration not required
>> [ 0.000000] percpu: Embedded 3 pages/cpu @fffffe007fda0000 s117832 r8192 d70584 u196608
>> [ 0.000000] Detected PIPT I-cache on CPU0
>> [ 0.000000] Built 2 zonelists in Node order, mobility grouping on. Total pages: 65472
>> [ 0.000000] Policy zone: Normal
>> [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.9.0-0.rc0.git7.1.fc26.aarch64 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap LANG=en_US.UTF-8 earlycon acpi=force
>> [ 0.000000] PID hash table entries: 4096 (order: -1, 32768 bytes)
>> [ 0.000000] software IO TLB [mem 0xdbff0000-0xdfff0000] (64MB) mapped at [fffffe009bff0000-fffffe009ffeffff]
>> [ 0.000000] Memory: 3542976K/4194304K available (9148K kernel code, 1612K rwdata, 3776K rodata, 1600K init, 15899K bss, 127040K reserved, 524288K cma-reserved)
>> [ 0.000000] Virtual kernel memory layout:
>> [ 0.000000] modules : 0xfffffc0000000000 - 0xfffffc0008000000 ( 128 MB)
>> vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000 ( 2045 GB)
>> .text : 0xfffffc0008080000 - 0xfffffc0008970000 ( 9152 KB)
>> .rodata : 0xfffffc0008970000 - 0xfffffc0008d30000 ( 3840 KB)
>> .init : 0xfffffc0008d30000 - 0xfffffc0008ec0000 ( 1600 KB)
>> .data : 0xfffffc0008ec0000 - 0xfffffc0009053200 ( 1613 KB)
>> .bss : 0xfffffc0009053200 - 0xfffffc0009fda058 ( 15900 KB)
>> fixed : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000 ( 4288 KB)
>> PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000 ( 16 MB)
>> vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000 ( 2 GB maximum)
>> 0xfffffdff80000000 - 0xfffffdff80400000 ( 4 MB actual)
>> memory : 0xfffffe0000000000 - 0xfffffe0100000000 ( 4096 MB)
>> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=2
>> [ 0.000000] Running RCU self tests
>> [ 0.000000] Hierarchical RCU implementation.
>> [ 0.000000] RCU lockdep checking is enabled.
>> [ 0.000000] Build-time adjustment of leaf fanout to 64.
>> [ 0.000000] RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=4.
>> [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=4
>> [ 0.000000] kmemleak: Kernel memory leak detector disabled
>> [ 0.000000] NR_IRQS:64 nr_irqs:64 0
>> [ 0.000000] GICv2m: ACPI overriding V2M MSI_TYPER (base:80, num:64)
>> [ 0.000000] GICv2m: range[mem 0x08020000-0x08020fff], SPI[80:143]
>> [ 0.000000] GIC: PPI11 is secure or misconfigured
>> [ 0.000000] arm_arch_timer: WARNING: Invalid trigger for IRQ3, assuming level low
>> [ 0.000000] arm_arch_timer: WARNING: Please fix your firmware
>> [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 50.00MHz (virt).
>> [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
>> [ 0.000003] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
>> [ 0.002198] Console: colour dummy device 80x25
>> [ 0.003319] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
>> [ 0.005236] ... MAX_LOCKDEP_SUBCLASSES: 8
>> [ 0.006183] ... MAX_LOCK_DEPTH: 48
>> [ 0.007273] ... MAX_LOCKDEP_KEYS: 8191
>> [ 0.008287] ... CLASSHASH_SIZE: 4096
>> [ 0.009296] ... MAX_LOCKDEP_ENTRIES: 32768
>> [ 0.010327] ... MAX_LOCKDEP_CHAINS: 65536
>> [ 0.011318] ... CHAINHASH_SIZE: 32768
>> [ 0.012453] memory used by lock dependency info: 8159 kB
>> [ 0.013736] per task-struct memory footprint: 1920 bytes
>> [ 0.015742] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
>> [ 0.018710] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=50000)
>> [ 0.021221] pid_max: default: 32768 minimum: 301
>> [ 0.022806] ACPI: Core revision 20160831
>> [ 0.027885] ACPI: 1 ACPI AML tables successfully acquired and loaded
>>
>> [ 0.030252] Security Framework initialized
>> [ 0.031355] Yama: becoming mindful.
>> [ 0.032176] SELinux: Initializing.
>> [ 0.033925] Dentry cache hash table entries: 524288 (order: 6, 4194304 bytes)
>> [ 0.037039] Inode-cache hash table entries: 262144 (order: 5, 2097152 bytes)
>> [ 0.039383] Mount-cache hash table entries: 8192 (order: 0, 65536 bytes)
>> [ 0.041135] Mountpoint-cache hash table entries: 8192 (order: 0, 65536 bytes)
>> [ 0.044725] ftrace: allocating 29596 entries in 8 pages
>> [ 0.080467] ASID allocator initialised with 65536 entries
>> [ 0.082070] ------------[ cut here ]------------
>> [ 0.083227] WARNING: CPU: 0 PID: 1 at kernel/workqueue.c:5458 wq_numa_init+0x178/0x21c
>> [ 0.085304] Modules linked in:
>> [ 0.086102]
>> [ 0.086499] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.9.0-0.rc0.git7.1.fc26.aarch64 #1
>> [ 0.088611] Hardware name: linux,dummy-virt (DT)
>> [ 0.089816] task: fffffe00700aac00 task.stack: fffffe00f8044000
>> [ 0.091375] PC is at wq_numa_init+0x178/0x21c
>> [ 0.092514] LR is at wq_numa_init+0x14c/0x21c
>> [ 0.093654] pc : [<fffffc0008d3f434>] lr : [<fffffc0008d3f408>] pstate: 60000045
>> [ 0.095589] sp : fffffe00f8047cb0
>> [ 0.096457] x29: fffffe00f8047cb0 [ 0.097311] x28: 0000000000000000
>> [ 0.098201]
>> [ 0.098601] x27: 0000000000000000 [ 0.099450] x26: fffffc0008ef4a28
>> [ 0.100342]
>> [ 0.100730] x25: fffffc0008ef3000 [ 0.101576] x24: fffffc0008ef3574
>> [ 0.102466]
>> [ 0.102853] x23: 0000000000000000 [ 0.103700] x22: fffffe007937de00
>> [ 0.104593]
>> [ 0.104982] x21: fffffc0008e887f8 [ 0.105829] x20: fffffc0009091000
>> [ 0.106723]
>> [ 0.107111] x19: 0000000000000000 [ 0.107956] x18: 0000000050642c6a
>> [ 0.108847]
>> [ 0.109234] x17: 0000000000000000 [ 0.110078] x16: 0000000000000000
>> [ 0.110968]
>> [ 0.111363] x15: 00000000fcacdc89 [ 0.112199] x14: 0000000000000000
>> [ 0.113087]
>> [ 0.113481] x13: 0000000000000000 [ 0.114324] x12: 00000000fe2ce6e0
>> [ 0.115204]
>> [ 0.115597] x11: 0000000000000001 [ 0.116439] x10: 0000000000000048
>> [ 0.117328]
>> [ 0.117716] x9 : 0000000000000000 [ 0.118563] x8 : fffffe00f4010080
>> [ 0.119453]
>> [ 0.119833] x7 : 0000000000000000 [ 0.120678] x6 : 0000000000000000
>> [ 0.121571]
>> [ 0.121959] x5 : 000000000000000f [ 0.122804] x4 : 0000000000000000
>> [ 0.123695]
>> [ 0.124084] x3 : 0000000000000000 [ 0.124922] x2 : 0000000000000000
>> [ 0.125815]
>> [ 0.126204] x1 : 0000000000000004 [ 0.127055] x0 : 00000000ffffffff
>> [ 0.127966]
>> [ 0.128361]
>> [ 0.128767] ---[ end trace 0000000000000000 ]---
>> [ 0.129983] Call trace:
>> [ 0.130629] Exception stack(0xfffffe00f8047ad0 to 0xfffffe00f8047c00)
>> [ 0.132316] 7ac0: 0000000000000000 0000040000000000
>> [ 0.134360] 7ae0: fffffe00f8047cb0 fffffc0008d3f434 0000000060000045 000000000000003d
>> [ 0.136405] 7b00: fffffc0008ef4000 fffffe007937df00 0000000000000000 0000000000000000
>> [ 0.138446] 7b20: fffffc0008bf4110 0000000000000189 0000000000000018 0000000000000028
>> [ 0.140498] 7b40: fffffe00f8047b80 0000000000000000 fffffe0000000000 fffffc000848af30
>> [ 0.142541] 7b60: fffffe00f8047ba0 fffffc0008134d24 fffffe00f8044000 0000000000000040
>> [ 0.144558] 7b80: 00000000ffffffff 0000000000000004 0000000000000000 0000000000000000
>> [ 0.146607] 7ba0: 0000000000000000 000000000000000f 0000000000000000 0000000000000000
>> [ 0.148664] 7bc0: fffffe00f4010080 0000000000000000 0000000000000048 0000000000000001
>> [ 0.150704] 7be0: 00000000fe2ce6e0 0000000000000000 0000000000000000 00000000fcacdc89
>> [ 0.152752] [<fffffc0008d3f434>] wq_numa_init+0x178/0x21c
>> [ 0.154160] [<fffffc0008d3f578>] init_workqueues+0xa0/0x4b8
>> [ 0.155596] [<fffffc0008083594>] do_one_initcall+0x44/0x138
>> [ 0.157059] [<fffffc0008d30d28>] kernel_init_freeable+0x178/0x2dc
>> [ 0.158670] [<fffffc0008956f48>] kernel_init+0x18/0x110
>> [ 0.160036] [<fffffc0008083330>] ret_from_fork+0x10/0x20
>> [ 0.161440] workqueue: NUMA node mapping not available for cpu0, disabling NUMA support
>> [ 0.165296] Remapping and enabling EFI services.
>> [ 0.166586] Unable to handle kernel paging request at virtual address b91000006be8
>> [ 0.168448] pgd = fffffc000a010000
>> [ 0.169341] [b91000006be8] *pgd=0000000000000000[ 0.170505] , *pud=0000000000000000
>> , *pmd=0000000000000000[ 0.171942]
>> [ 0.172332] Internal error: Oops: 96000004 [#1] SMP
>> [ 0.173600] Modules linked in:
>> [ 0.174407] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G W 4.9.0-0.rc0.git7.1.fc26.aarch64 #1
>> [ 0.176836] Hardware name: linux,dummy-virt (DT)
>> [ 0.178038] task: fffffe00700aac00 task.stack: fffffe00f8044000
>> [ 0.179579] PC is at __ll_sc_atomic_add+0x20/0x40
>> [ 0.180800] LR is at __lock_acquire+0xe8/0x698
>> [ 0.181961] pc : [<fffffc0008487390>] lr : [<fffffc0008138c08>] pstate: 800000c5
>> [ 0.183895] sp : fffffe00f8047820
>> [ 0.184755] x29: fffffe00f8047820 [ 0.185588] x28: fffffc0008ef3000
>> [ 0.186479]
>> [ 0.186868] x27: fffffc0008ef2358 [ 0.187713] x26: fffffc0009ce6000
>> [ 0.188606]
>> [ 0.188997] x25: 0000000000000001 [ 0.189857] x24: 0000000000000000
>> [ 0.190731]
>> [ 0.191115] x23: fffffe00700aac00 [ 0.191951] x22: 0000000000000000
>> [ 0.192843]
>> [ 0.193231] x21: fffffe007fd9a018 [ 0.194074] x20: 0000000000000000
>> [ 0.194966]
>> [ 0.195361] x19: fffffe007fd9a018 [ 0.196192] x18: 0000000000000010
>> [ 0.197077]
>> [ 0.197476] x17: 0000000057181979 [ 0.198325] x16: 0000000000000000
>> [ 0.199209]
>> [ 0.199604] x15: 0000000000000000 [ 0.200450] x14: 0000000000000000
>> [ 0.201337]
>> [ 0.201723] x13: 0000000000000001 [ 0.202555] x12: fffffe007fff2580
>> [ 0.203432]
>> [ 0.203819] x11: 0000000000000000 [ 0.204664] x10: 0000000000000011
>> [ 0.205550]
>> [ 0.205937] x9 : 0000000000000001 [ 0.206784] x8 : 0000b91000006be8
>> [ 0.207678]
>> [ 0.208062] x7 : fffffc0008299fcc [ 0.208899] x6 : 0000000000000000
>> [ 0.209787]
>> [ 0.210176] x5 : 0000000000000080 [ 0.211022] x4 : 0000b91000006a50
>> [ 0.211913]
>> [ 0.212307] x3 : 0000000000000000 [ 0.213147] x2 : 000022c80000f420
>> [ 0.214034]
>> [ 0.214421] x1 : 0000b91000006be8 [ 0.215251] x0 : fffffc0008138c08
>> [ 0.216134]
>> [ 0.216527]
>> [ 0.216916] Process swapper/0 (pid: 1, stack limit = 0xfffffe00f8044020)
>> [ 0.218671] Stack: (0xfffffe00f8047820 to 0xfffffe00f8048000)
>> [ 0.220167] 7820: fffffe00f8047840 fffffc0008138c08 fffffe00f8044000 0000000000000001
>> [ 0.222190] 7840: fffffe00f80478c0 fffffc0008139590 fffffe007fd9a018 0000000000000000
>> [ 0.224238] 7860: 0000000000000000 0000000000000000 0000000000000001 0000000000000000
>> [ 0.226284] 7880: fffffc0008299fcc 00000000000000c0 fffffc0008ef2358 fffffc0008ef3000
>> [ 0.228318] 78a0: 0000000000000001 fffffc0009ce6000 0000000000000000 fffffe0000000000
>> [ 0.230362] 78c0: fffffe00f8047930 fffffc000895f2c4 fffffe007fd9a000 fffffc0008299fcc
>> [ 0.232394] 78e0: fffffe007fd9a000 fffffc000829ad94 fffffe007001db00 000000000000e8e8
>> [ 0.234435] 7900: fffffe007001db00 fffffe007001dbf8 fffffe00fff3ef50 0000000000000000
>> [ 0.236481] 7920: fffffe00f8047a20 fffffc0008ef2000 fffffe00f8047950 fffffc0008299fcc
>> [ 0.238516] 7940: 00000000ffffffff fffffe007fd9a000 fffffe00f8047a70 fffffc000829aa68
>> [ 0.240560] 7960: 00000000ffffffff 0000000000000001 00000000024000c0 fffffc000829ad94
>> [ 0.242604] 7980: 0000000000210d00 000000000000e8e8 fffffe007001db00 fffffe007001dbf8
>> [ 0.244634] 79a0: fffffe00fff3ef50 0000000000000000 fffffe00f8044000 0000000000000040
>> [ 0.246678] 79c0: fffffc000828d620 fffffc0008ef3000 00000000026080c0 fffffe00fff3ef60
>> [ 0.248733] 79e0: fffffe00f8047a00 fffffc00024000c0 fffffc0008f89000 0000000000000000
>> [ 0.250783] 7a00: fffffe00f8047a20 fffffc000822f62c fffffc0009016b30 fffffe00f8047b40
>> [ 0.252896] 7a20: fffffe00f8047ba0 fffffc000828d620 0000000000000000 fffffc0008ef0b28
>> [ 0.255009] 7a40: fffffe007fff3c00 0000000000000000 0000000000000000 0000000000000000
>> [ 0.257121] 7a60: fffffe00f8044000 0000000000000000 fffffe00f8047b90 fffffc000829ad94
>> [ 0.259240] 7a80: 0000000000000040 fffffe007001db00 00000000024000c0 00000000ffffffff
>> [ 0.261358] 7aa0: fffffc0008266284 fffffe00fff3ef50 0000000020000000 00e8000000000f07
>> [ 0.263472] 7ac0: 0000000000000000 0000000000000400 fffffc0008f89000 0000000000000000
>> [ 0.265662] 7ae0: fffffe00f8047b00 fffffc000822f62c fffffe00fff3ef60 0000000000000000
>> [ 0.267787] 7b00: 0000001000000000 fffffc0008266284 fffffe00f8047b50 fffffc0008134d24
>> [ 0.269905] 7b20: fffffe00f8044000 0000000000000040 fffffc0008bf4110 0000000000000189
>> [ 0.272020] 7b40: fffffc0008ef4000 0000000000000000 fffffe00f8047b70 fffffc000810267c
>> [ 0.274136] 7b60: fffffc0009016893 0000000000000000 fffffe00f8047ba0 fffffc0008102784
>> [ 0.276250] 7b80: fffffe00f8047b90 fffffc000829ad7c fffffe00f8047bd0 fffffc000829b13c
>> [ 0.278371] 7ba0: fffffe007001db00 00000000024000c0 fffffc0008266284 fffffe007001db00
>> [ 0.280484] 7bc0: fffffc0008ef4000 0000000000000000 fffffe00f8047c30 fffffc0008266284
>> [ 0.282600] 7be0: fffffdff801b0200 fffffe006c080000 000000006c080000 0000000020000000
>> [ 0.284715] 7c00: fffffe00f0010008 0000000004000000 0000000020000000 00e8000000000f07
>> [ 0.286831] 7c20: 0000000000000000 0000000000000000 fffffe00f8047c50 fffffc0008098e24
>> [ 0.288948] 7c40: fffffdff801b0200 0000000000000001 fffffe00f8047c80 fffffc00080991d0
>> [ 0.291062] 7c60: 0000000024000000 0000000000000001 0000000024000000 fffffc0008ef0b28
>> [ 0.293178] 7c80: fffffe00f8047d00 fffffc0008d361cc fffffe0078416018 00e8000000000707
>> [ 0.295296] 7ca0: fffffc0008ff6410 fffffc0008ef7000 0000000000000000 fffffc0008ff6410
>> [ 0.297408] 7cc0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.299523] 7ce0: 0000000000000000 00e8000000000f05 fffffc0008098dd0 0000000023ffffff
>> [ 0.301636] 7d00: fffffe00f8047d10 fffffc0008d35020 fffffe00f8047d40 fffffc0008d88284
>> [ 0.303748] 7d20: fffffe0078416018 fffffc0008ff6000 fffffc0008c87348 fffffc0008d8821c
>> [ 0.305863] 7d40: fffffe00f8047d90 fffffc0008083594 fffffc0008d88154 fffffe00f8044000
>> [ 0.307987] 7d60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.310099] 7d80: 0000000000000000 0000000004000000 fffffe00f8047e00 fffffc0008d30d28
>> [ 0.312217] 7da0: fffffc0008e622d8 fffffc0008e622e0 0000000000000040 0000000000000000
>> [ 0.314333] 7dc0: fffffe00f8047e00 fffffc0008d30d18 fffffc0008e62220 fffffc0008e622e0
>> [ 0.316445] 7de0: 0000000000000040 0000000000000000 0000000000000000 fffffc0008e622e0
>> [ 0.318572] 7e00: fffffe00f8047ea0 fffffc0008956f48 fffffc0008956f30 0000000000000000
>> [ 0.320692] 7e20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.322805] 7e40: 0000000000000000 0000000000000000 0000000000000000 0000000000000001
>> [ 0.324914] 7e60: 0000000000000003 0000000000000000 0000000000000000 0000000000000000
>> [ 0.327027] 7e80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.329139] 7ea0: 0000000000000000 fffffc0008083330 fffffc0008956f30 0000000000000000
>> [ 0.331248] 7ec0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.333361] 7ee0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.335470] 7f00: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.337585] 7f20: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.339695] 7f40: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.341810] 7f60: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.343923] 7f80: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.346037] 7fa0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.348154] 7fc0: 0000000000000000 0000000000000005 0000000000000000 0000000000000000
>> [ 0.350272] 7fe0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
>> [ 0.352392] Call trace:
>> [ 0.353049] Exception stack(0xfffffe00f8047650 to 0xfffffe00f8047780)
>> [ 0.354792] 7640: fffffe007fd9a018 0000040000000000
>> [ 0.356910] 7660: fffffe00f8047820 fffffc0008487390 fffffe00f80476e0 fffffc0008131290
>> [ 0.359025] 7680: fffffc000901690b fffffc0008f1e000 0000000000000001 fffffe00700aac00
>> [ 0.361140] 76a0: fffffc000901690b fffffc0008f27a28 fffffe00fff3b700 fffffc0008e8b700
>> [ 0.363255] 76c0: fffffe00fff3b700 fffffc0008ef1000 fffffe00f80476e0 00000000000000c0
>> [ 0.365373] 76e0: fffffe00f8047720 fffffc000811a374 fffffc0008138c08 0000b91000006be8
>> [ 0.367483] 7700: 000022c80000f420 0000000000000000 0000b91000006a50 0000000000000080
>> [ 0.369593] 7720: 0000000000000000 fffffc0008299fcc 0000b91000006be8 0000000000000001
>> [ 0.371702] 7740: 0000000000000011 0000000000000000 fffffe007fff2580 0000000000000001
>> [ 0.373817] 7760: 0000000000000000 0000000000000000 0000000000000000 0000000057181979
>> [ 0.375935] [<fffffc0008487390>] __ll_sc_atomic_add+0x20/0x40
>> [ 0.377489] [<fffffc0008138c08>] __lock_acquire+0xe8/0x698
>> [ 0.378960] [<fffffc0008139590>] lock_acquire+0xd8/0x2c0
>> [ 0.380394] [<fffffc000895f2c4>] _raw_spin_lock+0x4c/0x60
>> [ 0.381843] [<fffffc0008299fcc>] get_partial_node.isra.23+0x4c/0x440
>> [ 0.383559] [<fffffc000829aa68>] ___slab_alloc+0x438/0x710
>> [ 0.385031] [<fffffc000829ad94>] __slab_alloc+0x54/0xa0
>> [ 0.386441] [<fffffc000829b13c>] kmem_cache_alloc+0x35c/0x428
>> [ 0.387983] [<fffffc0008266284>] ptlock_alloc+0x2c/0x58
>> [ 0.389394] [<fffffc0008098e24>] pgd_pgtable_alloc+0x54/0xd8
>> [ 0.390912] [<fffffc00080991d0>] __create_pgd_mapping+0x158/0x2a8
>> [ 0.392556] [<fffffc0008d361cc>] create_pgd_mapping+0x30/0x38
>> [ 0.394100] [<fffffc0008d35020>] efi_create_mapping+0xfc/0x110
>> [ 0.395682] [<fffffc0008d88284>] arm_enable_runtime_services+0x130/0x204
>> [ 0.397501] [<fffffc0008083594>] do_one_initcall+0x44/0x138
>> [ 0.399001] [<fffffc0008d30d28>] kernel_init_freeable+0x178/0x2dc
>> [ 0.400646] [<fffffc0008956f48>] kernel_init+0x18/0x110
>> [ 0.402053] [<fffffc0008083330>] ret_from_fork+0x10/0x20
>> [ 0.403488] Code: aa1e03e0 aa0103e8 d503201f f9800111 (885f7d00)
>> [ 0.405145] ---[ end trace f6be31446b0a9526 ]---
>> [ 0.406286] note: swapper/0[1] exited with preempt_count 1
>> [ 0.407687] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>> [ 0.407687]
>> [ 0.410047] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
>> [ 0.410047]
>>
>
> This log contains two call traces. The first is a WARNING in
> wq_numa_init(). The second is the unhandled page fault.
>
> Note the warning message (from wq_numa_init()):
>
> workqueue: NUMA node mapping not available for cpu0, disabling NUMA support
>
> Something looks genuinely broken with the cpu <-> numa-node
> associations in the ACPI case -- it even seems to fail when the SRAT
> does exist.
>
> So, perhaps, commit 7ba5f605f3a0 may not have introduced the bug, only
> exposed one in the ACPI code?...
Okay, so let me repeat,
smp_init_cpus() [arch/arm64/kernel/smp.c]
acpi_table_parse_madt() [drivers/acpi/tables.c]
acpi_parse_gic_cpu_interface() [arch/arm64/kernel/smp.c]
acpi_map_gic_cpu_interface() [arch/arm64/kernel/smp.c]
early_map_cpu_to_node() [arch/arm64/mm/numa.c]
We have acpi_map_gic_cpu_interface() being called for each GICC
structure in the MADT (signature "APIC"). This function is supposed to
set up a number of things for the CPU found, including its association
with a NUMA node. This should happen even if we have only one node (no
SRAT), and it should happen for CPU#0 as well.
acpi_map_gic_cpu_interface() uses the global variable "cpu_count" like
this:
(a) on input, it is the number of CPUs found previously, that is, the
logical identifier of the CPU being added presently,
(b) on output, it is bumped by one, if the CPU got added / parsed
correctly,
(c) in-between, we have expressions like:
> if (is_mpidr_duplicate(cpu_count, hwid)) {
> pr_err("duplicate CPU MPIDR 0x%llx in MADT\n", hwid);
> return;
> }
and
> if (cpu_count >= NR_CPUS)
> return;
(note: this implies that NR_CPUS is an exclusive limit)
and -- importantly --
> /* map the logical cpu id to cpu MPIDR */
> cpu_logical_map(cpu_count) = hwid;
and -- even more importantly --
> early_map_cpu_to_node(cpu_count, acpi_numa_get_nid(cpu_count, hwid));
A whole bunch of stuff seems to be wrong with this, when we try to
interpret it for CPU#0. Such as:
(1) the global variable "cpu_count" is initialized to one, not zero.
This dates back to the following commit:
> commit 0f0783365cbb7ec13a8f02198f6e1a146d94a5a9
> Author: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com>
> Date: Wed May 13 14:12:47 2015 +0100
>
> ARM64: kernel: unify ACPI and DT cpus initialization
It means that none of the above checks and assignments will be performed
for CPU#0.
It also means that should we actually find NR_CPUs CPUs, the last one
will be rejected, because at that point, cpu_count will equal NR_CPUs
*on input*.
(2) On arm64, cpu_logical_map() is implemented like this
[arch/arm64/include/asm/smp_plat.h]:
> /*
> * Logical CPU mapping.
> */
> extern u64 __cpu_logical_map[NR_CPUS];
> #define cpu_logical_map(cpu) __cpu_logical_map[cpu]
So this is the declaration. The definition is back in
"arch/arm64/kernel/setup.c":
> u64 __cpu_logical_map[NR_CPUS] = { [0 ... NR_CPUS-1] = INVALID_HWID };
where INVALID_HWID is ULONG_MAX.
This implies that
> /* map the logical cpu id to cpu MPIDR */
> cpu_logical_map(cpu_count) = hwid;
will never store a hwid different from INVALID_HWID to
__cpu_logical_map[0], because "cpu_count" -- the offset into that array,
for the assignment -- is never zero.
(3) early_map_cpu_to_node() will never set cpu_to_node_map[0] to any
NUMA node ID.
(If early_map_cpu_to_node() was called with cpu_count==0 (correctly), it
would call set_cpu_numa_node(), due to the change implemented by
7ba5f605f3a0:
> /*
> * We should set the numa node of cpu0 as soon as possible, because it
> * has already been set up online before. cpu_to_node(0) will soon be
> * called.
> */
> if (!cpu)
> set_cpu_numa_node(cpu, nid);
but I don't know what that would suffice for.)
(4) The acpi_numa_get_nid() function deserves separate treatment:
> int acpi_numa_get_nid(unsigned int cpu, u64 hwid)
> {
> int i;
>
> for (i = 0; i < cpus_in_srat; i++) {
> if (hwid == early_node_cpu_hwid[i].cpu_hwid)
> return early_node_cpu_hwid[i].node_id;
> }
>
> return NUMA_NO_NODE;
> }
So,
(4a) if we have no SRAT (because there's only one NUMA node), then this
function will invariably return NUMA_NO_NODE (value -1), which means
that *even if* early_map_cpu_to_node() was called with cpu_count==0
(which it is not, see (3) above), the assigned NUMA node ID would still
be NUMA_NO_NODE. That's wrong, it should be zero.
(4b) The acpi_numa_get_nid() function completely ignores its first
parameter, called "cpu" (set from "cpu_count" at the call site). This
has been the case since the birth of that function, namely
> commit d8b47fca8c233642d1a20fa4025579ebc8be6f1e
> Author: Hanjun Guo <hanjun.guo@linaro.org>
> Date: Tue May 24 15:35:44 2016 -0700
>
> arm64, ACPI, NUMA: NUMA support based on SRAT and SLIT
I guess if that parameter is unnecessary, it should be removed.
I'm sorry but I can't even begin to untangle this mess. Maybe the code I
tried to analyze in this email was never *meant* to associate CPU#0 with
any NUMA node at all (not even node 0); instead, other code -- for
example code removed by 7ba5f605f3a0 -- was meant to perform that
association.
If that's the case, then the code I listed here might even be correct,
for CPUs with logical IDs >= 1. The initialization of "cpu_count" to 1
does suggest that CPU#0 was never meant to be handled by
acpi_map_gic_cpu_interface(). I can't tell.
What I can tell is that 7ba5f605f3a0 breaks the ACPI boot. So
- either (parts of) it should be reverted please,
- or the ACPI boot path should be extended please, so that it handles
CPU#0 as well (associating it with NUMA node #0 if there is no SRAT,
and NUMA node #whatever, if there's an SRAT saying so).
Thanks,
Laszlo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 15:01 ` Laszlo Ersek
@ 2016-10-14 15:27 ` Laszlo Ersek
2016-10-14 15:42 ` Lorenzo Pieralisi
0 siblings, 1 reply; 11+ messages in thread
From: Laszlo Ersek @ 2016-10-14 15:27 UTC (permalink / raw)
To: Andrew Jones, Zhen Lei, Will Deacon, Lorenzo Pieralisi, Hanjun Guo
Cc: main kernel list, linux-arm-kernel, Ard Biesheuvel, Shannon Zhao,
Wei Huang
On 10/14/16 17:01, Laszlo Ersek wrote:
> Maybe the code I
> tried to analyze in this email was never *meant* to associate CPU#0 with
> any NUMA node at all (not even node 0); instead, other code -- for
> example code removed by 7ba5f605f3a0 -- was meant to perform that
> association.
Staring a bit more at the code, this looks very likely; in acpi_map_gic_cpu_interface() we have
> /* Check if GICC structure of boot CPU is available in the MADT */
> if (cpu_logical_map(0) == hwid) {
> if (bootcpu_valid) {
> pr_err("duplicate boot CPU MPIDR: 0x%llx in MADT\n",
> hwid);
> return;
> }
> bootcpu_valid = true;
> return;
> }
which means that this callback function (for parsing the GICC structures in the MADT) expects to find the boot processor as well.
Upon finding the boot processor, we set bootcpu_valid to true, and that's it -- no association with any NUMA node, and no incrementing of "cpu_count".
Thanks
Laszlo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 15:27 ` Laszlo Ersek
@ 2016-10-14 15:42 ` Lorenzo Pieralisi
2016-10-14 16:22 ` Laszlo Ersek
0 siblings, 1 reply; 11+ messages in thread
From: Lorenzo Pieralisi @ 2016-10-14 15:42 UTC (permalink / raw)
To: Laszlo Ersek
Cc: Andrew Jones, Zhen Lei, Will Deacon, Hanjun Guo,
main kernel list, linux-arm-kernel, Ard Biesheuvel, Shannon Zhao,
Wei Huang
On Fri, Oct 14, 2016 at 05:27:58PM +0200, Laszlo Ersek wrote:
> On 10/14/16 17:01, Laszlo Ersek wrote:
>
> > Maybe the code I
> > tried to analyze in this email was never *meant* to associate CPU#0 with
> > any NUMA node at all (not even node 0); instead, other code -- for
> > example code removed by 7ba5f605f3a0 -- was meant to perform that
> > association.
>
> Staring a bit more at the code, this looks very likely; in acpi_map_gic_cpu_interface() we have
>
> > /* Check if GICC structure of boot CPU is available in the MADT */
> > if (cpu_logical_map(0) == hwid) {
> > if (bootcpu_valid) {
> > pr_err("duplicate boot CPU MPIDR: 0x%llx in MADT\n",
> > hwid);
> > return;
> > }
> > bootcpu_valid = true;
> > return;
> > }
>
> which means that this callback function (for parsing the GICC
> structures in the MADT) expects to find the boot processor as well.
>
> Upon finding the boot processor, we set bootcpu_valid to true, and
> that's it -- no association with any NUMA node, and no incrementing of
> "cpu_count".
Yes, because that's to check the MADT contains the boot cpu hwid.
Does this help (compile tested only) ?
-- >8 --
diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
index d3f151c..8507703 100644
--- a/arch/arm64/kernel/smp.c
+++ b/arch/arm64/kernel/smp.c
@@ -544,6 +544,7 @@ static int __init smp_cpu_setup(int cpu)
return;
}
bootcpu_valid = true;
+ early_map_cpu_to_node(0, acpi_numa_get_nid(0, hwid));
return;
}
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 13:44 ` Andrew Jones
@ 2016-10-14 15:45 ` Laszlo Ersek
0 siblings, 0 replies; 11+ messages in thread
From: Laszlo Ersek @ 2016-10-14 15:45 UTC (permalink / raw)
To: Andrew Jones
Cc: Zhen Lei, Will Deacon, main kernel list, linux-arm-kernel,
Ard Biesheuvel, Shannon Zhao, Wei Huang
On 10/14/16 15:44, Andrew Jones wrote:
> On Fri, Oct 14, 2016 at 03:18:00PM +0200, Laszlo Ersek wrote:
>> On 10/14/16 10:05, Andrew Jones wrote:
>>> On Fri, Oct 14, 2016 at 12:50:29AM +0200, Laszlo Ersek wrote:
>>>> (4) Analysis (well, a lame attempt at that, because I have zero
>>>> familiarity with this code). Let me quote the patch:
>>>>
>>>>> commit 7ba5f605f3a0d9495aad539eeb8346d726dfc183
>>>>> Author: Zhen Lei <thunder.leizhen@huawei.com>
>>>>> Date: Thu Sep 1 14:55:04 2016 +0800
>>>>>
>>>>> arm64/numa: remove the limitation that cpu0 must bind to node0
>>>>>
>>>>> 1. Remove the old binding code.
>>>>> 2. Read the nid of cpu0 from dts.
>>>>> 3. Fallback the nid of cpu0 to 0 when numa=off is set in bootargs.
>>>>>
>>>>> Signed-off-by: Zhen Lei <thunder.leizhen@huawei.com>
>>>>> Signed-off-by: Will Deacon <will.deacon@arm.com>
>>>>>
>>>>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>>>>> index c3c08368a685..8b048e6ec34a 100644
>>>>> --- a/arch/arm64/kernel/smp.c
>>>>> +++ b/arch/arm64/kernel/smp.c
>>>>> @@ -624,6 +624,7 @@ static void __init of_parse_and_init_cpus(void)
>>>>> }
>>>>>
>>>>> bootcpu_valid = true;
>>>>> + early_map_cpu_to_node(0, of_node_to_nid(dn));
>>>>>
>>>>> /*
>>>>> * cpu_logical_map has already been
>>>>> diff --git a/arch/arm64/mm/numa.c b/arch/arm64/mm/numa.c
>>>>> index 0a15f010b64a..778a985c8a70 100644
>>>>> --- a/arch/arm64/mm/numa.c
>>>>> +++ b/arch/arm64/mm/numa.c
>>>>> @@ -116,16 +116,24 @@ static void __init setup_node_to_cpumask_map(void)
>>>>> */
>>>>> void numa_store_cpu_info(unsigned int cpu)
>>>>> {
>>>>> - map_cpu_to_node(cpu, numa_off ? 0 : cpu_to_node_map[cpu]);
>>>>> + map_cpu_to_node(cpu, cpu_to_node_map[cpu]);
>>>>> }
>>>>>
>>>>> void __init early_map_cpu_to_node(unsigned int cpu, int nid)
>>>>> {
>>>>> /* fallback to node 0 */
>>>>> - if (nid < 0 || nid >= MAX_NUMNODES)
>>>>> + if (nid < 0 || nid >= MAX_NUMNODES || numa_off)
>>>>> nid = 0;
>>>
>>> The ACPI equivalent code must be missing (at least) the above, because,
>>> even with DT, mach-virt won't have cpu to node mappings unless numa
>>> is configured on the command line. Can you try adding something like
>>>
>>> -m 512 -smp 4 \
>>> -numa node,mem=256M,cpus=0-1,nodeid=0 \
>>> -numa node,mem=256M,cpus=2-3,nodeid=1
>>>
>>> to your QEMU command line?
>>
>> I added the following to my domain XML, under <cpu>:
>>
>> <numa>
>> <cell id='0' cpus='0-1' memory='2097152' unit='KiB'/>
>> <cell id='1' cpus='2-3' memory='2097152' unit='KiB'/>
>> </numa>
>>
>> (See <http://libvirt.org/formatdomain.html#elementsCPU>.)
>>
>> With that, each NUMA node gets half of the VCPUs and half of the guest RAM.
>>
>> (This is in a different guest now, one that has a bleeding edge Fedora kernel -- I didn't want to rebuild the upstream kernel yet again, just for this test. So, "4.9.0-0.rc0.git7.1.fc26.aarch64" is based on upstream v4.8-14109-g1573d2c, and it reproduces the problem too.)
>>
>>> Then when you boot with ACPI you'll get a
>>> SRAT.
>>
>> Yes, that's confirmed by the guest kernel log (see below).
>>
>>> If that works, then we're just missing the "no SRAT, nid = 0"
>>> code (that should have been added with this patch)
>>
>> It still crashes with the SRAT, with the following log:
>
> Rats.
>
>> Note the warning message (from wq_numa_init()):
>>
>> workqueue: NUMA node mapping not available for cpu0, disabling NUMA support
>>
>> Something looks genuinely broken with the cpu <-> numa-node associations in the ACPI case -- it even seems to fail when the SRAT does exist.
>>
>> So, perhaps, commit 7ba5f605f3a0 may not have introduced the bug, only exposed one in the ACPI code?...
>
> The kernel's ACPI NUMA support used to work. It was the test case for
> QEMU's SRAT generation code.
>
> Two more experiments I wouldn't mind having you try, if you have time;
> 1) confirm that this NUMA configured guest and this latest kernel works
> with DT. This is just for sanity, but I guess it will.
Yes, it boots fine. It's at v4.8-14604-g29fbff8.
> 2) If (1) succeeds, then try the last-known-good kernel with ACPI again,
> but this time with the NUMA configured guest.
v4.8, which was my known-good starting point in the acpi=force bisection -- at that time without using a NUMA configuration --, also boots fine under this two-node NUMA configuration, using acpi=force:
> EFI stub: Booting Linux Kernel...
> EFI stub: Using DTB from configuration table
> EFI stub: Exiting boot services and installing virtual address map...
> [ 0.000000] Booting Linux on physical CPU 0x0
> [ 0.000000] Linux version 4.8.0 (root@aarch64-vgpu-1) (gcc version 6.2.1 20160916 (Red Hat 6.2.1-2) (GCC) ) #2 SMP Thu Oct 13 17:17:59 CEST 2016
> [ 0.000000] Boot CPU: AArch64 Processor [500f0000]
> [ 0.000000] earlycon: pl11 at MMIO 0x0000000009000000 (options '')
> [ 0.000000] bootconsole [pl11] enabled
> [ 0.000000] debug: ignoring loglevel setting.
> [ 0.000000] efi: Getting EFI parameters from FDT:
> [ 0.000000] efi: EFI v2.60 by EDK II
> [ 0.000000] efi: SMBIOS 3.0=0x13bdb0000 ACPI 2.0=0x1386d0000 MEMATTR=0x13a665018
> [ 0.000000] cma: Reserved 512 MiB at 0x00000000c0000000
> [ 0.000000] ACPI: Early table checksum verification disabled
> [ 0.000000] ACPI: RSDP 0x00000001386D0000 000024 (v02 BOCHS )
> [ 0.000000] ACPI: XSDT 0x00000001386C0000 000054 (v01 BOCHS BXPCFACP 00000001 01000013)
> [ 0.000000] ACPI: FACP 0x0000000138410000 00010C (v05 BOCHS BXPCFACP 00000001 BXPC 00000001)
> [ 0.000000] ACPI: DSDT 0x0000000138420000 001159 (v02 BOCHS BXPCDSDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: APIC 0x0000000138400000 0002BC (v03 BOCHS BXPCAPIC 00000001 BXPC 00000001)
> [ 0.000000] ACPI: GTDT 0x00000001383F0000 000060 (v02 BOCHS BXPCGTDT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: MCFG 0x00000001383E0000 00003C (v01 BOCHS BXPCMCFG 00000001 BXPC 00000001)
> [ 0.000000] ACPI: SPCR 0x00000001383D0000 000050 (v02 BOCHS BXPCSPCR 00000001 BXPC 00000001)
> [ 0.000000] ACPI: SRAT 0x00000001383C0000 000110 (v03 BOCHS BXPCSRAT 00000001 BXPC 00000001)
> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x0 -> Node 0
> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x1 -> Node 0
> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x2 -> Node 0
> [ 0.000000] ACPI: NUMA: SRAT: PXM 0 -> MPIDR 0x3 -> Node 0
> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x4 -> Node 1
> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x5 -> Node 1
> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x6 -> Node 1
> [ 0.000000] ACPI: NUMA: SRAT: PXM 1 -> MPIDR 0x7 -> Node 1
> [ 0.000000] NUMA: Adding memblock [0x40000000 - 0x13fffffff] on node 0
> [ 0.000000] ACPI: SRAT: Node 0 PXM 0 [mem 0x40000000-0x13fffffff]
> [ 0.000000] NUMA: Adding memblock [0x140000000 - 0x23fffffff] on node 1
> [ 0.000000] ACPI: SRAT: Node 1 PXM 1 [mem 0x140000000-0x23fffffff]
> [ 0.000000] NUMA: Initmem setup node 0 [mem 0x40000000-0x13fffffff]
> [ 0.000000] NUMA: NODE_DATA [mem 0x13fff2580-0x13fffffff]
> [ 0.000000] NUMA: Initmem setup node 1 [mem 0x140000000-0x23fffffff]
> [ 0.000000] NUMA: NODE_DATA [mem 0x23fff2580-0x23fffffff]
> [ 0.000000] Zone ranges:
> [ 0.000000] DMA [mem 0x0000000040000000-0x00000000ffffffff]
> [ 0.000000] Normal [mem 0x0000000100000000-0x000000023fffffff]
> [ 0.000000] Movable zone start for each node
> [ 0.000000] Early memory node ranges
> [ 0.000000] node 0: [mem 0x0000000040000000-0x00000000fffeffff]
> [ 0.000000] node 0: [mem 0x00000000ffff0000-0x00000000ffffffff]
> [ 0.000000] node 0: [mem 0x0000000100000000-0x00000001383bffff]
> [ 0.000000] node 0: [mem 0x00000001383c0000-0x000000013874ffff]
> [ 0.000000] node 0: [mem 0x0000000138750000-0x000000013bc1ffff]
> [ 0.000000] node 0: [mem 0x000000013bc20000-0x000000013bffffff]
> [ 0.000000] node 0: [mem 0x000000013c000000-0x000000013fffffff]
> [ 0.000000] node 1: [mem 0x0000000140000000-0x000000023fffffff]
> [ 0.000000] Initmem setup node 0 [mem 0x0000000040000000-0x000000013fffffff]
> [ 0.000000] On node 0 totalpages: 65536
> [ 0.000000] DMA zone: 48 pages used for memmap
> [ 0.000000] DMA zone: 0 pages reserved
> [ 0.000000] DMA zone: 49152 pages, LIFO batch:1
> [ 0.000000] Normal zone: 16 pages used for memmap
> [ 0.000000] Normal zone: 16384 pages, LIFO batch:1
> [ 0.000000] Initmem setup node 1 [mem 0x0000000140000000-0x000000023fffffff]
> [ 0.000000] On node 1 totalpages: 65536
> [ 0.000000] Normal zone: 64 pages used for memmap
> [ 0.000000] Normal zone: 65536 pages, LIFO batch:1
> [ 0.000000] psci: probing for conduit method from ACPI.
> [ 0.000000] psci: PSCIv0.2 detected in firmware.
> [ 0.000000] psci: Using standard PSCI v0.2 function IDs
> [ 0.000000] psci: Trusted OS migration not required
> [ 0.000000] percpu: Embedded 3 pages/cpu @fffffe01ffdc0000 s117320 r8192 d71096 u196608
> [ 0.000000] pcpu-alloc: s117320 r8192 d71096 u196608 alloc=3*65536
> [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [0] 4 [0] 5 [0] 6 [0] 7
> [ 0.000000] Detected PIPT I-cache on CPU0
> [ 0.000000] Built 2 zonelists in Node order, mobility grouping on. Total pages: 130944
> [ 0.000000] Policy zone: Normal
> [ 0.000000] Kernel command line: BOOT_IMAGE=/vmlinuz-4.8.0 root=/dev/mapper/fedora-root ro rd.lvm.lv=fedora/root rd.lvm.lv=fedora/swap console=ttyAMA0 earlyprintk=pl011,0x9000000 earlycon ignore_loglevel acpi=force
> [ 0.000000] PID hash table entries: 4096 (order: -1, 32768 bytes)
> [ 0.000000] software IO TLB [mem 0xfbfe0000-0xfffe0000] (64MB) mapped at [fffffe00bbfe0000-fffffe00bffdffff]
> [ 0.000000] Memory: 7717184K/8388608K available (8956K kernel code, 1565K rwdata, 3712K rodata, 1536K init, 15876K bss, 147136K reserved, 524288K cma-reserved)
> [ 0.000000] Virtual kernel memory layout:
> [ 0.000000] modules : 0xfffffc0000000000 - 0xfffffc0008000000 ( 128 MB)
> [ 0.000000] vmalloc : 0xfffffc0008000000 - 0xfffffdff5fff0000 ( 2045 GB)
> [ 0.000000] .text : 0xfffffc0008080000 - 0xfffffc0008940000 ( 8960 KB)
> [ 0.000000] .rodata : 0xfffffc0008940000 - 0xfffffc0008cf0000 ( 3776 KB)
> [ 0.000000] .init : 0xfffffc0008cf0000 - 0xfffffc0008e70000 ( 1536 KB)
> [ 0.000000] .data : 0xfffffc0008e70000 - 0xfffffc0008ff7400 ( 1565 KB)
> [ 0.000000] .bss : 0xfffffc0008ff7400 - 0xfffffc0009f78628 ( 15877 KB)
> [ 0.000000] fixed : 0xfffffdff7e7d0000 - 0xfffffdff7ec00000 ( 4288 KB)
> [ 0.000000] PCI I/O : 0xfffffdff7ee00000 - 0xfffffdff7fe00000 ( 16 MB)
> [ 0.000000] vmemmap : 0xfffffdff80000000 - 0xfffffe0000000000 ( 2 GB maximum)
> [ 0.000000] 0xfffffdff80000000 - 0xfffffdff80800000 ( 8 MB actual)
> [ 0.000000] memory : 0xfffffe0000000000 - 0xfffffe0200000000 ( 8192 MB)
> [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=8, Nodes=2
> [ 0.000000] Running RCU self tests
> [ 0.000000] Hierarchical RCU implementation.
> [ 0.000000] RCU lockdep checking is enabled.
> [ 0.000000] Build-time adjustment of leaf fanout to 64.
> [ 0.000000] RCU restricting CPUs from NR_CPUS=256 to nr_cpu_ids=8.
> [ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=8
> [ 0.000000] kmemleak: Kernel memory leak detector disabled
> [ 0.000000] NR_IRQS:64 nr_irqs:64 0
> [ 0.000000] GICv2m: ACPI overriding V2M MSI_TYPER (base:80, num:64)
> [ 0.000000] GICv2m: range[mem 0x08020000-0x08020fff], SPI[80:143]
> [ 0.000000] GIC: PPI11 is secure or misconfigured
> [ 0.000000] arm_arch_timer: WARNING: Invalid trigger for IRQ3, assuming level low
> [ 0.000000] arm_arch_timer: WARNING: Please fix your firmware
> [ 0.000000] arm_arch_timer: Architected cp15 timer(s) running at 50.00MHz (virt).
> [ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb8812736b, max_idle_ns: 440795202655 ns
> [ 0.000002] sched_clock: 56 bits at 50MHz, resolution 20ns, wraps every 4398046511100ns
> [ 0.002263] Console: colour dummy device 80x25
> [ 0.003385] Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> [ 0.005291] ... MAX_LOCKDEP_SUBCLASSES: 8
> [ 0.006233] ... MAX_LOCK_DEPTH: 48
> [ 0.007335] ... MAX_LOCKDEP_KEYS: 8191
> [ 0.008339] ... CLASSHASH_SIZE: 4096
> [ 0.009334] ... MAX_LOCKDEP_ENTRIES: 32768
> [ 0.010497] ... MAX_LOCKDEP_CHAINS: 65536
> [ 0.011517] ... CHAINHASH_SIZE: 32768
> [ 0.012541] memory used by lock dependency info: 8159 kB
> [ 0.013810] per task-struct memory footprint: 1920 bytes
> [ 0.015809] mempolicy: Enabling automatic NUMA balancing. Configure with numa_balancing= or the kernel.numa_balancing sysctl
> [ 0.018762] Calibrating delay loop (skipped), value calculated using timer frequency.. 100.00 BogoMIPS (lpj=50000)
> [ 0.021262] pid_max: default: 32768 minimum: 301
> [ 0.022804] ACPI: Core revision 20160422
> [ 0.027895] ACPI: 1 ACPI AML tables successfully acquired and loaded
> [ 0.029485]
> [ 0.030614] Security Framework initialized
> [ 0.031599] Yama: becoming mindful.
> [ 0.032422] SELinux: Initializing.
> [ 0.033949] SELinux: Starting in permissive mode
> [ 0.035498] Dentry cache hash table entries: 1048576 (order: 7, 8388608 bytes)
> [ 0.039287] Inode-cache hash table entries: 524288 (order: 6, 4194304 bytes)
> [ 0.042406] Mount-cache hash table entries: 16384 (order: 1, 131072 bytes)
> [ 0.044138] Mountpoint-cache hash table entries: 16384 (order: 1, 131072 bytes)
> [ 0.047820] ftrace: allocating 28926 entries in 8 pages
> [ 0.082086] ASID allocator initialised with 65536 entries
> [ 0.086231] Remapping and enabling EFI services.
> [ 0.087582] EFI remap 0x0000000004000000 => 0000000020000000
> [ 0.089058] EFI remap 0x0000000009010000 => 0000000024000000
> [ 0.090486] EFI remap 0x0000000138430000 => 0000000024010000
> [ 0.091877] EFI remap 0x0000000138490000 => 0000000024070000
> [ 0.093269] EFI remap 0x00000001384e0000 => 00000000240c0000
> [ 0.094687] EFI remap 0x0000000138530000 => 0000000024110000
> [ 0.096076] EFI remap 0x0000000138580000 => 0000000024160000
> [ 0.097471] EFI remap 0x00000001385d0000 => 00000000241b0000
> [ 0.098855] EFI remap 0x0000000138620000 => 0000000024200000
> [ 0.100243] EFI remap 0x0000000138670000 => 0000000024250000
> [ 0.101638] EFI remap 0x00000001386e0000 => 00000000242a0000
> [ 0.103022] EFI remap 0x000000013bc20000 => 0000000024310000
> [ 0.104420] EFI remap 0x000000013bdb0000 => 00000000244a0000
> [ 0.120433] Detected PIPT I-cache on CPU1
> [ 0.120539] CPU1: Booted secondary processor [500f0000]
> [ 0.134831] Detected PIPT I-cache on CPU2
> [ 0.134918] CPU2: Booted secondary processor [500f0000]
> [ 0.149136] Detected PIPT I-cache on CPU3
> [ 0.149226] CPU3: Booted secondary processor [500f0000]
> [ 0.163487] Detected PIPT I-cache on CPU4
> [ 0.163576] CPU4: Booted secondary processor [500f0000]
> [ 0.177837] Detected PIPT I-cache on CPU5
> [ 0.177926] CPU5: Booted secondary processor [500f0000]
> [ 0.192172] Detected PIPT I-cache on CPU6
> [ 0.192260] CPU6: Booted secondary processor [500f0000]
> [ 0.206722] Detected PIPT I-cache on CPU7
> [ 0.206811] CPU7: Booted secondary processor [500f0000]
> [ 0.207003] Brought up 8 CPUs
> [ 0.223904] SMP: Total of 8 processors activated.
> [ 0.225110] CPU features: detected feature: 32-bit EL0 Support
> [ 0.226534] CPU: All CPU(s) started at EL1
> [ 0.319814] devtmpfs: initialized
> [ 0.321977] SMBIOS 3.0.0 present.
> [ 0.323394] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
> [ 0.331773] atomic64_test: passed
> [ 0.332749] pinctrl core: initialized pinctrl subsystem
> [ 0.335763] NET: Registered protocol family 16
> [ 0.342141] cpuidle: using governor menu
> [ 0.343192] PCCT header not found.
> [ 0.344082] vdso: 2 pages (1 code @ fffffc0008960000, 1 data @ fffffc0008e90000)
> [ 0.346114] hw-breakpoint: found 4 breakpoint and 4 watchpoint registers.
> [ 0.349683] DMA: preallocated 256 KiB pool for atomic allocations
> [ 0.351473] ACPI: bus type PCI registered
> [ 0.352483] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
> [ 0.354595] Serial: AMBA PL011 UART driver
> [ 0.373196] HugeTLB registered 2 MB page size, pre-allocated 0 pages
> [ 0.374902] HugeTLB registered 512 MB page size, pre-allocated 0 pages
> [ 0.380062] ACPI: Added _OSI(Module Device)
> [ 0.381157] ACPI: Added _OSI(Processor Device)
> [ 0.382200] ACPI: Added _OSI(3.0 _SCP Extensions)
> [ 0.383303] ACPI: Added _OSI(Processor Aggregator Device)
> [ 0.392501] ACPI: Interpreter enabled
> [ 0.393484] ACPI: Using GIC for interrupt routing
> [ 0.394751] ACPI: MCFG table detected, 1 entries
> [ 0.414519] ARMH0011:00: ttyAMA0 at MMIO 0x9000000 (irq = 5, base_baud = 0) is a SBSA
> [ 0.416591] console [ttyAMA0] enabled
> [ 0.416591] console [ttyAMA0] enabled
> [ 0.418231] bootconsole [pl11] disabled
> [ 0.418231] bootconsole [pl11] disabled
> [ 0.428624] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-0f])
> [ 0.429962] acpi PNP0A08:00: _OSC: OS supports [ExtendedConfig ASPM ClockPM Segments MSI]
> [ 0.432827] acpi PNP0A08:00: _OSC: OS now controls [PCIeHotplug PME AER PCIeCapability]
> [ 0.434477] acpi PNP0A08:00: ECAM at [mem 0x3f000000-0x3fffffff] for [bus 00-0f]
> [ 0.436078] Remapped I/O 0x000000003eff0000 to [io 0x0000-0xffff window]
> [ 0.438129] PCI host bridge to bus 0000:00
> [ 0.438987] pci_bus 0000:00: root bus resource [mem 0x10000000-0x3efeffff window]
> [ 0.440462] pci_bus 0000:00: root bus resource [io 0x0000-0xffff window]
> [ 0.441750] pci_bus 0000:00: root bus resource [mem 0x8000000000-0xffffffffff window]
> [ 0.443252] pci_bus 0000:00: root bus resource [bus 00-0f]
> [ 0.444377] pci 0000:00:00.0: [1b36:0008] type 00 class 0x060000
> [ 0.446402] pci 0000:00:01.0: [8086:3420] type 01 class 0x060400
> [ 0.489217] pci 0000:00:01.1: [8086:3420] type 01 class 0x060400
> [ 0.528741] pci 0000:00:01.2: [8086:3420] type 01 class 0x060400
> [ 0.567928] pci 0000:00:01.3: [8086:3420] type 01 class 0x060400
> [ 0.613634] pci 0000:01:00.0: [1033:0194] type 00 class 0x0c0330
> [ 0.614948] pci 0000:01:00.0: reg 0x10: [mem 0x10300000-0x10303fff 64bit]
> [ 0.631100] pci 0000:02:00.0: [1af4:1048] type 00 class 0x010000
> [ 0.679413] pci 0000:02:00.0: reg 0x14: [mem 0x10200000-0x10200fff]
> [ 0.752358] pci 0000:02:00.0: reg 0x20: [mem 0x8000000000-0x80007fffff 64bit pref]
> [ 0.793032] pci 0000:03:00.0: [1af4:1041] type 00 class 0x020000
> [ 0.841238] pci 0000:03:00.0: reg 0x14: [mem 0x10100000-0x10100fff]
> [ 0.914350] pci 0000:03:00.0: reg 0x20: [mem 0x8000800000-0x8000ffffff 64bit pref]
> [ 0.939819] pci 0000:03:00.0: reg 0x30: [mem 0xfffc0000-0xffffffff pref]
> [ 0.956386] pci 0000:04:00.0: [1af4:1050] type 00 class 0x038000
> [ 1.004617] pci 0000:04:00.0: reg 0x14: [mem 0x10000000-0x10000fff]
> [ 1.077643] pci 0000:04:00.0: reg 0x20: [mem 0x8001000000-0x80017fffff 64bit pref]
> [ 1.213423] pci 0000:00:01.1: BAR 15: assigned [mem 0x8000000000-0x80007fffff 64bit pref]
> [ 1.215037] pci 0000:00:01.2: BAR 15: assigned [mem 0x8000800000-0x8000ffffff 64bit pref]
> [ 1.216626] pci 0000:00:01.3: BAR 15: assigned [mem 0x8001000000-0x80017fffff 64bit pref]
> [ 1.219273] pci 0000:00:01.0: BAR 14: assigned [mem 0x10000000-0x101fffff]
> [ 1.220663] pci 0000:00:01.0: BAR 15: assigned [mem 0x8001800000-0x80019fffff 64bit pref]
> [ 1.222265] pci 0000:00:01.1: BAR 14: assigned [mem 0x10200000-0x103fffff]
> [ 1.223605] pci 0000:00:01.2: BAR 14: assigned [mem 0x10400000-0x105fffff]
> [ 1.224945] pci 0000:00:01.3: BAR 14: assigned [mem 0x10600000-0x107fffff]
> [ 1.226270] pci 0000:00:01.0: BAR 13: assigned [io 0x1000-0x1fff]
> [ 1.227475] pci 0000:00:01.1: BAR 13: assigned [io 0x2000-0x2fff]
> [ 1.228668] pci 0000:00:01.2: BAR 13: assigned [io 0x3000-0x3fff]
> [ 1.229853] pci 0000:00:01.3: BAR 13: assigned [io 0x4000-0x4fff]
> [ 1.231064] pci 0000:01:00.0: BAR 0: assigned [mem 0x10000000-0x10003fff 64bit]
> [ 1.232564] pci 0000:00:01.0: PCI bridge to [bus 01]
> [ 1.233530] pci 0000:00:01.0: bridge window [io 0x1000-0x1fff]
> [ 1.253214] pci 0000:00:01.0: bridge window [mem 0x10000000-0x101fffff]
> [ 1.267540] pci 0000:00:01.0: bridge window [mem 0x8001800000-0x80019fffff 64bit pref]
> [ 1.294996] pci 0000:02:00.0: BAR 4: assigned [mem 0x8000000000-0x80007fffff 64bit pref]
> [ 1.321082] pci 0000:02:00.0: BAR 1: assigned [mem 0x10200000-0x10200fff]
> [ 1.323053] pci 0000:00:01.1: PCI bridge to [bus 02]
> [ 1.324004] pci 0000:00:01.1: bridge window [io 0x2000-0x2fff]
> [ 1.344149] pci 0000:00:01.1: bridge window [mem 0x10200000-0x103fffff]
> [ 1.358181] pci 0000:00:01.1: bridge window [mem 0x8000000000-0x80007fffff 64bit pref]
> [ 1.384572] pci 0000:03:00.0: BAR 4: assigned [mem 0x8000800000-0x8000ffffff 64bit pref]
> [ 1.410293] pci 0000:03:00.0: BAR 6: assigned [mem 0x10400000-0x1043ffff pref]
> [ 1.412715] pci 0000:03:00.0: BAR 1: assigned [mem 0x10440000-0x10440fff]
> [ 1.426032] pci 0000:00:01.2: PCI bridge to [bus 03]
> [ 1.427117] pci 0000:00:01.2: bridge window [io 0x3000-0x3fff]
> [ 1.446229] pci 0000:00:01.2: bridge window [mem 0x10400000-0x105fffff]
> [ 1.460070] pci 0000:00:01.2: bridge window [mem 0x8000800000-0x8000ffffff 64bit pref]
> [ 1.486803] pci 0000:04:00.0: BAR 4: assigned [mem 0x8001000000-0x80017fffff 64bit pref]
> [ 1.512454] pci 0000:04:00.0: BAR 1: assigned [mem 0x10600000-0x10600fff]
> [ 1.526310] pci 0000:00:01.3: PCI bridge to [bus 04]
> [ 1.527283] pci 0000:00:01.3: bridge window [io 0x4000-0x4fff]
> [ 1.546456] pci 0000:00:01.3: bridge window [mem 0x10600000-0x107fffff]
> [ 1.560368] pci 0000:00:01.3: bridge window [mem 0x8001000000-0x80017fffff 64bit pref]
> [ 1.587389] ACPI: PCI Interrupt Link [GSI0] (IRQs *35)
> [ 1.588527] ACPI: PCI Interrupt Link [GSI1] (IRQs *36)
> [ 1.589607] ACPI: PCI Interrupt Link [GSI2] (IRQs *37)
> [ 1.590670] ACPI: PCI Interrupt Link [GSI3] (IRQs *38)
> [ 1.594331] vgaarb: loaded
> [ 1.595642] SCSI subsystem initialized
> [ 1.596943] libata version 3.00 loaded.
> [ 1.597907] ACPI: bus type USB registered
> [ 1.598884] usbcore: registered new interface driver usbfs
> [ 1.600034] usbcore: registered new interface driver hub
> [ 1.601350] usbcore: registered new device driver usb
> [ 1.604990] NetLabel: Initializing
> [ 1.605716] NetLabel: domain hash size = 128
> [ 1.606524] NetLabel: protocols = UNLABELED CIPSOv4
> [ 1.607669] NetLabel: unlabeled traffic allowed by default
> [ 1.611142] DMA-API: preallocated 4096 debug entries
> [ 1.612105] DMA-API: debugging enabled by kernel config
> [ 1.705795] clocksource: Switched to clocksource arch_sys_counter
> [ 1.787821] VFS: Disk quotas dquot_6.6.0
> [ 1.789040] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
> [ 1.791938] pnp: PnP ACPI init
> [ 1.793312] pnp: PnP ACPI: found 0 devices
> [ 1.810253] NET: Registered protocol family 2
> [ 1.813094] TCP established hash table entries: 65536 (order: 3, 524288 bytes)
> [ 1.814846] TCP bind hash table entries: 65536 (order: 6, 4194304 bytes)
> [ 1.822396] TCP: Hash tables configured (established 65536 bind 65536)
> [ 1.823873] UDP hash table entries: 4096 (order: 3, 655360 bytes)
> [ 1.825936] UDP-Lite hash table entries: 4096 (order: 3, 655360 bytes)
> [ 1.828672] NET: Registered protocol family 1
> [ 1.829673] pci 0000:01:00.0: enabling device (0000 -> 0002)
> [ 1.837906] PCI: CLS 0 bytes, default 128
> [ 1.839209] Unpacking initramfs...
> [ 2.755342] Freeing initrd memory: 30400K (fffffe00f53c0000 - fffffe00f7170000)
> [ 2.757373] kvm [1]: HYP mode not available
> [ 2.758651] alg: No test for __ecb-aes-neon (__driver-ecb-aes-neon)
> [ 2.760171] cryptomgr_test (75) used greatest stack depth: 13824 bytes left
> [ 2.767725] modprobe (81) used greatest stack depth: 12944 bytes left
> [ 2.772040] alg: No test for __ecb-aes-neon (cryptd(__driver-ecb-aes-neon))
> [ 2.774374] cryptomgr_test (79) used greatest stack depth: 12368 bytes left
> [ 2.777350] futex hash table entries: 2048 (order: 2, 262144 bytes)
> [ 2.779027] audit: initializing netlink subsys (disabled)
> [ 2.780231] audit: type=2000 audit(2.600:1): initialized
> [ 2.782020] Initialise system trusted keyrings
> [ 2.784150] workingset: timestamp_bits=37 max_order=17 bucket_order=0
> [ 2.802041] zbud: loaded
> [ 2.808395] Key type big_key registered
> [ 2.809229] SELinux: Registering netfilter hooks
> [ 2.897252] alg: drbg: Test 0 failed for drbg_pr_ctr_aes128
> [ 2.898554] cryptomgr_test (104) used greatest stack depth: 12016 bytes left
> [ 2.908631] alg: drbg: Test 0 failed for drbg_nopr_ctr_aes128
> [ 2.910204] alg: drbg: Test 0 failed for drbg_nopr_ctr_aes192
> [ 2.911772] alg: drbg: Test 0 failed for drbg_nopr_ctr_aes256
> [ 2.915338] NET: Registered protocol family 38
> [ 2.916380] Key type asymmetric registered
> [ 2.917318] Asymmetric key parser 'x509' registered
> [ 2.918805] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
> [ 2.920707] io scheduler noop registered
> [ 2.921443] io scheduler deadline registered
> [ 2.923122] io scheduler cfq registered (default)
> [ 2.924139] start plist test
> [ 2.927415] end plist test
> [ 2.931670] pl061_gpio ARMH0061:00: PL061 GPIO chip @0x0000000009030000 registered
> [ 2.934507] ACPI: PCI Interrupt Link [GSI1] enabled at IRQ 36
> [ 2.973148] aer 0000:00:01.0:pcie002: service driver aer loaded
> [ 2.974725] aer 0000:00:01.1:pcie002: service driver aer loaded
> [ 2.976977] aer 0000:00:01.2:pcie002: service driver aer loaded
> [ 2.978502] aer 0000:00:01.3:pcie002: service driver aer loaded
> [ 2.979878] pcieport 0000:00:01.0: Signaling PME through PCIe PME interrupt
> [ 2.981335] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
> [ 2.982761] pcie_pme 0000:00:01.0:pcie001: service driver pcie_pme loaded
> [ 2.984298] pcieport 0000:00:01.1: Signaling PME through PCIe PME interrupt
> [ 2.985781] pci 0000:02:00.0: Signaling PME through PCIe PME interrupt
> [ 2.987087] pcie_pme 0000:00:01.1:pcie001: service driver pcie_pme loaded
> [ 2.988619] pcieport 0000:00:01.2: Signaling PME through PCIe PME interrupt
> [ 2.990050] pci 0000:03:00.0: Signaling PME through PCIe PME interrupt
> [ 2.991448] pcie_pme 0000:00:01.2:pcie001: service driver pcie_pme loaded
> [ 2.993019] pcieport 0000:00:01.3: Signaling PME through PCIe PME interrupt
> [ 2.994518] pci 0000:04:00.0: Signaling PME through PCIe PME interrupt
> [ 2.995867] pcie_pme 0000:00:01.3:pcie001: service driver pcie_pme loaded
> [ 2.997328] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
> [ 2.998614] pciehp 0000:00:01.0:pcie004: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- LLActRep-
> [ 3.001848] pciehp 0000:00:01.0:pcie004: service driver pciehp loaded
> [ 3.003202] pciehp 0000:00:01.1:pcie004: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- LLActRep-
> [ 3.006097] pciehp 0000:00:01.1:pcie004: service driver pciehp loaded
> [ 3.007533] pciehp 0000:00:01.2:pcie004: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- LLActRep-
> [ 3.010330] pciehp 0000:00:01.2:pcie004: service driver pciehp loaded
> [ 3.011748] pciehp 0000:00:01.3:pcie004: Slot #0 AttnBtn+ PwrCtrl+ MRL- AttnInd+ PwrInd+ HotPlug+ Surprise+ Interlock+ NoCompl- LLActRep-
> [ 3.014587] pciehp 0000:00:01.3:pcie004: service driver pciehp loaded
> [ 3.015864] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
> [ 3.017707] efifb: invalid framebuffer address
> [ 3.020376] Serial: 8250/16550 driver, 32 ports, IRQ sharing enabled
> [ 3.033876] msm_serial: driver initialized
> [ 3.035876] Failed to find cpu0 device node
> [ 3.036713] Unable to detect cache hierarchy from DT for CPU 0
> [ 3.038968] libphy: Fixed MDIO Bus: probed
> [ 3.040219] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
> [ 3.041636] ehci-pci: EHCI PCI platform driver
> [ 3.042554] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
> [ 3.043761] ohci-pci: OHCI PCI platform driver
> [ 3.044666] uhci_hcd: USB Universal Host Controller Interface driver
> [ 3.054963] xhci_hcd 0000:01:00.0: xHCI Host Controller
> [ 3.056725] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 1
> [ 3.060495] xhci_hcd 0000:01:00.0: hcc params 0x00087001 hci version 0x100 quirks 0x00000014
> [ 3.064236] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
> [ 3.065654] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> [ 3.067032] usb usb1: Product: xHCI Host Controller
> [ 3.067982] usb usb1: Manufacturer: Linux 4.8.0 xhci-hcd
> [ 3.069014] usb usb1: SerialNumber: 0000:01:00.0
> [ 3.071283] hub 1-0:1.0: USB hub found
> [ 3.072298] hub 1-0:1.0: 4 ports detected
> [ 3.074921] xhci_hcd 0000:01:00.0: xHCI Host Controller
> [ 3.076485] xhci_hcd 0000:01:00.0: new USB bus registered, assigned bus number 2
> [ 3.078304] usb usb2: We don't know the algorithms for LPM for this host, disabling LPM.
> [ 3.080160] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003
> [ 3.081526] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
> [ 3.082893] usb usb2: Product: xHCI Host Controller
> [ 3.083821] usb usb2: Manufacturer: Linux 4.8.0 xhci-hcd
> [ 3.084825] usb usb2: SerialNumber: 0000:01:00.0
> [ 3.086840] hub 2-0:1.0: USB hub found
> [ 3.087745] hub 2-0:1.0: 4 ports detected
> [ 3.090180] usbcore: registered new interface driver usbserial
> [ 3.091473] usbcore: registered new interface driver usbserial_generic
> [ 3.093048] usbserial: USB Serial support registered for generic
> [ 3.094847] mousedev: PS/2 mouse device common for all mice
> [ 3.097427] rtc-efi rtc-efi: rtc core: registered rtc-efi as rtc0
> [ 3.099602] device-mapper: uevent: version 1.0.3
> [ 3.101336] device-mapper: ioctl: 4.35.0-ioctl (2016-06-23) initialised: dm-devel@redhat.com
> [ 3.104534] EFI Variables Facility v0.08 2004-May-17
> [ 3.107690] hidraw: raw HID events driver (C) Jiri Kosina
> [ 3.109001] usbcore: registered new interface driver usbhid
> [ 3.110141] usbhid: USB HID core driver
> [ 3.111920] drop_monitor: Initializing network drop monitor service
> [ 3.113583] ip_tables: (C) 2000-2006 Netfilter Core Team
> [ 3.114948] Initializing XFRM netlink socket
> [ 3.117302] NET: Registered protocol family 10
> [ 3.120493] mip6: Mobile IPv6
> [ 3.121237] NET: Registered protocol family 17
> [ 3.123363] registered taskstats version 1
> [ 3.124289] Loading compiled-in X.509 certificates
> [ 3.132881] alg: No test for pkcs1pad(rsa,sha256) (pkcs1pad(rsa-generic,sha256))
> [ 3.136159] Loaded X.509 cert 'Build time autogenerated kernel key: 646bee5e17f26888c6e6ef5950998cb2000dfbae'
> [ 3.138723] zswap: loaded using pool lzo/zbud
> [ 3.166483] rtc-efi rtc-efi: setting system clock to 2016-10-14 15:41:17 UTC (1476459677)
> [ 3.168727] PM: Hibernation image not present or could not be loaded.
> [ 3.170839] Freeing unused kernel memory: 1536K (fffffe0000cf0000 - fffffe0000e70000)
> [ 3.190833] random: systemd: uninitialized urandom read (16 bytes read)
> [ 3.194327] random: systemd: uninitialized urandom read (16 bytes read)
> [ 3.213481] systemd[1]: systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN)
> [ 3.217655] systemd[1]: Detected virtualization kvm.
> [ 3.218614] systemd[1]: Detected architecture arm64.
> [ 3.219547] systemd[1]: Running in initial RAM disk.
>
> Welcome to Fedora 24 (Server Edition) dracut-044-20.fc24 (Initramfs)!
> If (2) fails then we may need to expand the bisection :-(
Thankfully, it didn't fail.
Thanks
Laszlo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 15:42 ` Lorenzo Pieralisi
@ 2016-10-14 16:22 ` Laszlo Ersek
2016-10-14 16:58 ` Lorenzo Pieralisi
2016-10-17 8:04 ` Leizhen (ThunderTown)
0 siblings, 2 replies; 11+ messages in thread
From: Laszlo Ersek @ 2016-10-14 16:22 UTC (permalink / raw)
To: Lorenzo Pieralisi
Cc: Andrew Jones, Zhen Lei, Will Deacon, Hanjun Guo,
main kernel list, linux-arm-kernel, Ard Biesheuvel, Shannon Zhao,
Wei Huang
On 10/14/16 17:42, Lorenzo Pieralisi wrote:
> On Fri, Oct 14, 2016 at 05:27:58PM +0200, Laszlo Ersek wrote:
>> On 10/14/16 17:01, Laszlo Ersek wrote:
>>
>>> Maybe the code I
>>> tried to analyze in this email was never *meant* to associate CPU#0 with
>>> any NUMA node at all (not even node 0); instead, other code -- for
>>> example code removed by 7ba5f605f3a0 -- was meant to perform that
>>> association.
>>
>> Staring a bit more at the code, this looks very likely; in acpi_map_gic_cpu_interface() we have
>>
>>> /* Check if GICC structure of boot CPU is available in the MADT */
>>> if (cpu_logical_map(0) == hwid) {
>>> if (bootcpu_valid) {
>>> pr_err("duplicate boot CPU MPIDR: 0x%llx in MADT\n",
>>> hwid);
>>> return;
>>> }
>>> bootcpu_valid = true;
>>> return;
>>> }
>>
>> which means that this callback function (for parsing the GICC
>> structures in the MADT) expects to find the boot processor as well.
>>
>> Upon finding the boot processor, we set bootcpu_valid to true, and
>> that's it -- no association with any NUMA node, and no incrementing of
>> "cpu_count".
>
> Yes, because that's to check the MADT contains the boot cpu hwid.
>
> Does this help (compile tested only) ?
>
> -- >8 --
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index d3f151c..8507703 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -544,6 +544,7 @@ static int __init smp_cpu_setup(int cpu)
> return;
> }
> bootcpu_valid = true;
> + early_map_cpu_to_node(0, acpi_numa_get_nid(0, hwid));
> return;
> }
>
>
Your patch applies to the tree at v4.8-14604-g29fbff8698fc, but the function the hunk modifies is not smp_cpu_setup(), it is acpi_map_gic_cpu_interface():
> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> index d3f151cfd4a1..8507703dabe4 100644
> --- a/arch/arm64/kernel/smp.c
> +++ b/arch/arm64/kernel/smp.c
> @@ -544,6 +544,7 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
> return;
> }
> bootcpu_valid = true;
> + early_map_cpu_to_node(0, acpi_numa_get_nid(0, hwid));
> return;
> }
>
Anyway, your patch works with both the two-node NUMA configuration Drew suggested for testing, and with the single-node config that I originally used for the bisection. Therefore:
Tested-by: Laszlo Ersek <lersek@redhat.com>
Reported-by: Laszlo Ersek <lersek@redhat.com>
Thank you very much for the quick bugfix! And, I think your patch (when you send it for real) should carry
Fixes: 7ba5f605f3a0d9495aad539eeb8346d726dfc183
too, because it supplies the cpu#0<->node#xxx association that 7ba5f605f3a0 removed not just for DT, but also for ACPI.
Cheers!
Laszlo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 16:22 ` Laszlo Ersek
@ 2016-10-14 16:58 ` Lorenzo Pieralisi
2016-10-17 8:04 ` Leizhen (ThunderTown)
1 sibling, 0 replies; 11+ messages in thread
From: Lorenzo Pieralisi @ 2016-10-14 16:58 UTC (permalink / raw)
To: Laszlo Ersek
Cc: Andrew Jones, Zhen Lei, Will Deacon, Hanjun Guo,
main kernel list, linux-arm-kernel, Ard Biesheuvel, Shannon Zhao,
Wei Huang
On Fri, Oct 14, 2016 at 06:22:55PM +0200, Laszlo Ersek wrote:
> On 10/14/16 17:42, Lorenzo Pieralisi wrote:
> > On Fri, Oct 14, 2016 at 05:27:58PM +0200, Laszlo Ersek wrote:
> >> On 10/14/16 17:01, Laszlo Ersek wrote:
> >>
> >>> Maybe the code I
> >>> tried to analyze in this email was never *meant* to associate CPU#0 with
> >>> any NUMA node at all (not even node 0); instead, other code -- for
> >>> example code removed by 7ba5f605f3a0 -- was meant to perform that
> >>> association.
> >>
> >> Staring a bit more at the code, this looks very likely; in acpi_map_gic_cpu_interface() we have
> >>
> >>> /* Check if GICC structure of boot CPU is available in the MADT */
> >>> if (cpu_logical_map(0) == hwid) {
> >>> if (bootcpu_valid) {
> >>> pr_err("duplicate boot CPU MPIDR: 0x%llx in MADT\n",
> >>> hwid);
> >>> return;
> >>> }
> >>> bootcpu_valid = true;
> >>> return;
> >>> }
> >>
> >> which means that this callback function (for parsing the GICC
> >> structures in the MADT) expects to find the boot processor as well.
> >>
> >> Upon finding the boot processor, we set bootcpu_valid to true, and
> >> that's it -- no association with any NUMA node, and no incrementing of
> >> "cpu_count".
> >
> > Yes, because that's to check the MADT contains the boot cpu hwid.
> >
> > Does this help (compile tested only) ?
> >
> > -- >8 --
> > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> > index d3f151c..8507703 100644
> > --- a/arch/arm64/kernel/smp.c
> > +++ b/arch/arm64/kernel/smp.c
> > @@ -544,6 +544,7 @@ static int __init smp_cpu_setup(int cpu)
> > return;
> > }
> > bootcpu_valid = true;
> > + early_map_cpu_to_node(0, acpi_numa_get_nid(0, hwid));
> > return;
> > }
> >
> >
>
> Your patch applies to the tree at v4.8-14604-g29fbff8698fc, but the function the hunk modifies is not smp_cpu_setup(), it is acpi_map_gic_cpu_interface():
>
> > diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
> > index d3f151cfd4a1..8507703dabe4 100644
> > --- a/arch/arm64/kernel/smp.c
> > +++ b/arch/arm64/kernel/smp.c
> > @@ -544,6 +544,7 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
> > return;
> > }
> > bootcpu_valid = true;
> > + early_map_cpu_to_node(0, acpi_numa_get_nid(0, hwid));
> > return;
> > }
> >
>
> Anyway, your patch works with both the two-node NUMA configuration
> Drew suggested for testing, and with the single-node config that I
> originally used for the bisection. Therefore:
>
> Tested-by: Laszlo Ersek <lersek@redhat.com>
> Reported-by: Laszlo Ersek <lersek@redhat.com>
>
> Thank you very much for the quick bugfix! And, I think your patch
> (when you send it for real) should carry
>
> Fixes: 7ba5f605f3a0d9495aad539eeb8346d726dfc183
>
> too, because it supplies the cpu#0<->node#xxx association that
> 7ba5f605f3a0 removed not just for DT, but also for ACPI.
Sure, will do, I will send it out on Monday.
Cheers,
Lorenzo
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0")
2016-10-14 16:22 ` Laszlo Ersek
2016-10-14 16:58 ` Lorenzo Pieralisi
@ 2016-10-17 8:04 ` Leizhen (ThunderTown)
1 sibling, 0 replies; 11+ messages in thread
From: Leizhen (ThunderTown) @ 2016-10-17 8:04 UTC (permalink / raw)
To: Laszlo Ersek, Lorenzo Pieralisi
Cc: Andrew Jones, Will Deacon, Hanjun Guo, main kernel list,
linux-arm-kernel, Ard Biesheuvel, Shannon Zhao, Wei Huang
>> diff --git a/arch/arm64/kernel/smp.c b/arch/arm64/kernel/smp.c
>> index d3f151cfd4a1..8507703dabe4 100644
>> --- a/arch/arm64/kernel/smp.c
>> +++ b/arch/arm64/kernel/smp.c
>> @@ -544,6 +544,7 @@ acpi_map_gic_cpu_interface(struct acpi_madt_generic_interrupt *processor)
>> return;
>> }
>> bootcpu_valid = true;
>> + early_map_cpu_to_node(0, acpi_numa_get_nid(0, hwid));
>> return;
>> }
>>
>
> Anyway, your patch works with both the two-node NUMA configuration Drew suggested for testing, and with the single-node config that I originally used for the bisection. Therefore:
>
> Tested-by: Laszlo Ersek <lersek@redhat.com>
> Reported-by: Laszlo Ersek <lersek@redhat.com>
>
> Thank you very much for the quick bugfix! And, I think your patch (when you send it for real) should carry
I'm so sorry about this. My patch series prepared before ACPI NUMA upstreamed, and forgot considering it in later.
>
> Fixes: 7ba5f605f3a0d9495aad539eeb8346d726dfc183
>
> too, because it supplies the cpu#0<->node#xxx association that 7ba5f605f3a0 removed not just for DT, but also for ACPI.
>
> Cheers!
> Laszlo
>
> .
>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-10-17 8:51 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-13 22:50 aarch64 ACPI boot regressed by commit 7ba5f605f3a0 ("arm64/numa: remove the limitation that cpu0 must bind to node0") Laszlo Ersek
2016-10-14 8:05 ` Andrew Jones
2016-10-14 13:18 ` Laszlo Ersek
2016-10-14 13:44 ` Andrew Jones
2016-10-14 15:45 ` Laszlo Ersek
2016-10-14 15:01 ` Laszlo Ersek
2016-10-14 15:27 ` Laszlo Ersek
2016-10-14 15:42 ` Lorenzo Pieralisi
2016-10-14 16:22 ` Laszlo Ersek
2016-10-14 16:58 ` Lorenzo Pieralisi
2016-10-17 8:04 ` Leizhen (ThunderTown)
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).