* Problems building and running Xen on Hikey960
@ 2018-11-08 7:07 Matthew Daley
2018-11-09 11:22 ` Julien Grall
0 siblings, 1 reply; 13+ messages in thread
From: Matthew Daley @ 2018-11-08 7:07 UTC (permalink / raw)
To: Xen-devel
[-- Attachment #1: Type: text/plain, Size: 6045 bytes --]
Hi all,
I'm trying to build and run Linux (Debian) on top of Xen on a Hikey960
ARM board but am having trouble.
I've been following Konrad's guide on the Wiki (at
https://wiki.xenproject.org/wiki/HiKey960), namely:
* I have upgraded the board's base firmware to support UEFI
* I am building the hikey960-upstream-rebase branch of
https://github.com/96boards-hikey/linux.git (with a patch from Linus's
tree to fix a Hynix UFS bug)
* I am building the master branch of Xen unstable
* I am building the master branch of GRUB (with a patch to fix a DTB
padding bug)
However, I see two problems when booting.
Firstly, Xen fails to bring up any other CPUs but the one it is booting on:
(XEN) Bringing up CPU1
(XEN) Failed to bring up CPU1
(XEN) Failed to bring up CPU 1 (error -9)
(XEN) Bringing up CPU2
(XEN) Failed to bring up CPU2
(XEN) Failed to bring up CPU 2 (error -9)
(XEN) Bringing up CPU3
(XEN) Failed to bring up CPU3
(XEN) Failed to bring up CPU 3 (error -9)
(XEN) Bringing up CPU4
(XEN) Failed to bring up CPU4
(XEN) Failed to bring up CPU 4 (error -9)
(XEN) Bringing up CPU5
(XEN) Failed to bring up CPU5
(XEN) Failed to bring up CPU 5 (error -9)
(XEN) Bringing up CPU6
(XEN) Failed to bring up CPU6
(XEN) Failed to bring up CPU 6 (error -9)
(XEN) Bringing up CPU7
(XEN) Failed to bring up CPU7
(XEN) Failed to bring up CPU 7 (error -9)
(XEN) Brought up 1 CPUs
I have traced this error code -9 being returned by call_psci_cpu_on.
Secondly, Linux fails when it tries to initialise AMBA devices:
[ 0.941352] Synchronous External Abort: synchronous external abort
(0x96000210) at 0xffff0000093fdfe
0
[ 0.950601] Internal error: : 96000210 [#1] PREEMPT SMP
[ 0.955866] Modules linked in:
[ 0.958990] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
4.14.0-rc7-linaro-hikey960+ #8
[ 0.966791] Hardware name: HiKey960 (DT)
[ 0.970777] task: ffff80001d900000 task.stack: ffff000008058000
[ 0.976778] PC is at amba_device_try_add+0x108/0x260
[ 0.981791] LR is at amba_device_try_add+0xf0/0x260
[ 0.986735] pc : [<ffff0000084eada8>] lr : [<ffff0000084ead90>]
pstate: 60000045
[ 0.994192] sp : ffff00000805bbf0
[ 0.997572] x29: ffff00000805bbf0 x28: 0000000000000000
[ 1.002953] x27: ffff0000090203a8 x26: 0000000000000000
[ 1.008326] x25: ffff80001dbf9810 x24: 0000000000000000
[ 1.013702] x23: ffff0000093fd000 x22: 0000000000001000
[ 1.019079] x21: ffff80001cc1b6f8 x20: 0000000000000000
[ 1.024455] x19: ffff80001cc1b400 x18: 0000000000000010
[ 1.029832] x17: 0000000000000001 x16: 00000000deadbeef
[ 1.035209] x15: 0000000000000006 x14: ffffffffffffffff
[ 1.040585] x13: 0000000000000020 x12: 0101010101010101
[ 1.045962] x11: 0000000000000020 x10: 0101010101010101
[ 1.051338] x9 : 0000000000000000 x8 : ffff80001cc0cf00
[ 1.056717] x7 : 0000000000000000 x6 : 000000000000003f
[ 1.062092] x5 : 0000000000000000 x4 : 0000000000000000
[ 1.067468] x3 : 0000000000000000 x2 : 0000000000000000
[ 1.072845] x1 : ffff80001d900000 x0 : ffff0000093fdfe0
[ 1.078223] Process swapper/0 (pid: 1, stack limit = 0xffff000008058000)
[ 1.084989] Call trace:
[ 1.087504] Exception stack(0xffff00000805bab0 to 0xffff00000805bbf0)
[ 1.094008] baa0:
ffff0000093fdfe0 ffff80001d900000
[ 1.101902] bac0: 0000000000000000 0000000000000000
0000000000000000 0000000000000000
[ 1.109790] bae0: 000000000000003f 0000000000000000
ffff80001cc0cf00 0000000000000000
[ 1.117684] bb00: 0101010101010101 0000000000000020
0101010101010101 0000000000000020
[ 1.125576] bb20: ffffffffffffffff 0000000000000006
00000000deadbeef 0000000000000001
[ 1.133468] bb40: 0000000000000010 ffff80001cc1b400
0000000000000000 ffff80001cc1b6f8
[ 1.141356] bb60: 0000000000001000 ffff0000093fd000
0000000000000000 ffff80001dbf9810
[ 1.149248] bb80: 0000000000000000 ffff0000090203a8
0000000000000000 ffff00000805bbf0
[ 1.157139] bba0: ffff0000084ead90 ffff00000805bbf0
ffff0000084eada8 0000000060000045
[ 1.165034] bbc0: ffff00000805bbf0 ffff0000084ead90
ffffffffffffffff 00000000fffffffe
[ 1.172921] bbe0: ffff00000805bbf0 ffff0000084eada8
[ 1.177865] [<ffff0000084eada8>] amba_device_try_add+0x108/0x260
[ 1.183935] [<ffff0000084eafec>] amba_device_add+0x1c/0xd8
[ 1.189493] [<ffff00000890fbbc>] of_platform_bus_create+0x26c/0x300
[ 1.195814] [<ffff00000890fa74>] of_platform_bus_create+0x124/0x300
[ 1.202145] [<ffff00000890fd7c>] of_platform_populate+0x4c/0xb0
[ 1.208135] [<ffff000008f6e2ac>] of_platform_default_populate_init+0x64/0x78
[ 1.215247] [<ffff000008083978>] do_one_initcall+0x38/0x120
[ 1.220882] [<ffff000008f20d18>] kernel_init_freeable+0x184/0x224
[ 1.227038] [<ffff000008a6a360>] kernel_init+0x10/0x100
[ 1.232323] [<ffff000008084b60>] ret_from_fork+0x10/0x18
[ 1.237703] Code: d10082c0 52800002 8b0002e0 52800018 (b9400001)
[ 1.243880] ---[ end trace dcbf70aa30c979a8 ]---
[ 1.248573] Kernel panic - not syncing: Attempted to kill init!
exitcode=0x0000000b
This does not occur when booting the same Linux kernel without Xen.
I have traced this to amba_device_try_add trying to access the pid and
cid through ioremap'd addresses of this node in the device tree (in
arch/arm64/boot/dts/hisilicon/hi3660-coresight.dtsi):
/* A73 cluster internal coresight */
etm@4,ed440000 {
compatible = "arm,coresight-etm4x","arm,primecell";
reg = <0 0xed440000 0 0x1000>;
clocks = <&pclk>;
clock-names = "apb_pclk";
cpu = <&cpu4>;
port {
etm4_out_port: endpoint {
remote-endpoint = <&funnel1_in_port0>;
};
};
};
ARM is still relatively new to me and I'm stuck what I should be
attempting next. I would simply not compile Linux AMBA support (by not
setting CONFIG_AMBA) but it appears that this is selected as a reverse
dependency by CONFIG_ARM on Linux, so am unsure if this is wise or
even possible.
I've attached the complete Xen + Linux serial logs.
Keen to hear any suggestions, hacky or otherwise.
- Matthew Daley
[-- Attachment #2: hikey960-xen.txt --]
[-- Type: text/plain, Size: 14511 bytes --]
Xen 4.12-unstable (c/s Fri Sep 7 11:41:31 2018 +0100 git:1d069e45f7) EFI loader
Xen 4.12-unstable
(XEN) Xen version 4.12-unstable () (aarch64-linux-gnu-gcc (Debian 6.3.0-18) 6.3.0 20170516) debug=y Tue Nov 6 00:45:38 NZDT 2018
(XEN) Latest ChangeSet: Fri Sep 7 11:41:31 2018 +0100 git:1d069e45f7
(XEN) Processor: 410fd034: "ARM Limited", variant: 0x0, part 0xd03, rev 0x4
(XEN) 64-bit Execution:
(XEN) Processor Features: 0000000000002222 0000000000000000
(XEN) Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN) Extensions: FloatingPoint AdvancedSIMD
(XEN) Debug Features: 0000000010305106 0000000000000000
(XEN) Auxiliary Features: 0000000000000000 0000000000000000
(XEN) Memory Model Features: 0000000000001122 0000000000000000
(XEN) ISA Features: 0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN) Processor Features: 00000131:00011011
(XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN) Extensions: GenericTimer Security
(XEN) Debug Features: 03010066
(XEN) Auxiliary Features: 00000000
(XEN) Memory Model Features: 10201105 40000000 01260000 02102211
(XEN) ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Using SMC Calling Convention v1.1
(XEN) Using PSCI v1.1
(XEN) SMP: Allowing 8 CPUs
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 1920 KHz
(XEN) GICv2 initialization:
(XEN) gic_dist_addr=00000000e82b1000
(XEN) gic_cpu_addr=00000000e82b2000
(XEN) gic_hyp_addr=00000000e82b4000
(XEN) gic_vcpu_addr=00000000e82b6000
(XEN) gic_maintenance_irq=25
(XEN) GICv2: 384 lines, 8 cpus, secure (IID 0200143b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 64 KiB.
(XEN) Bringing up CPU1
(XEN) Failed to bring up CPU1
(XEN) Failed to bring up CPU 1 (error -9)
(XEN) Bringing up CPU2
(XEN) Failed to bring up CPU2
(XEN) Failed to bring up CPU 2 (error -9)
(XEN) Bringing up CPU3
(XEN) Failed to bring up CPU3
(XEN) Failed to bring up CPU 3 (error -9)
(XEN) Bringing up CPU4
(XEN) Failed to bring up CPU4
(XEN) Failed to bring up CPU 4 (error -9)
(XEN) Bringing up CPU5
(XEN) Failed to bring up CPU5
(XEN) Failed to bring up CPU 5 (error -9)
(XEN) Bringing up CPU6
(XEN) Failed to bring up CPU6
(XEN) Failed to bring up CPU 6 (error -9)
(XEN) Bringing up CPU7
(XEN) Failed to bring up CPU7
(XEN) Failed to bring up CPU 7 (error -9)
(XEN) Brought up 1 CPUs
(XEN) P2M: 40-bit IPA with 40-bit PA and 8-bit VMID
(XEN) P2M: 3 levels with order-1 root, VTCR 0x80023558
(XEN) I/O virtualisation disabled
(XEN) build-id: 6cefdc2172cd6ea6e6e8bc90c7842d2cac723219
(XEN) alternatives: Patching with alt table 00000000002a3b40 -> 00000000002a40d4
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 00000000b8885000
(XEN) Allocating 1:1 mappings totalling 512MB for dom0:
(XEN) BANK[0] 0x000000c0000000-0x000000e0000000 (512MB)
(XEN) Grant table range: 0x0000021fe00000-0x0000021fe40000
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading zImage from 00000000b8885000 to 00000000c0080000-00000000c1167200
(XEN) Loading dom0 DTB to 0x00000000c8000000-0x00000000c800a4cf
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Scrubbing Free RAM on 1 nodes using 1 CPUs
(XEN) ....................................................................done.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) ***************************************************
(XEN) PLEASE SPECIFY dom0_mem PARAMETER - USING 512M FOR NOW
(XEN) ***************************************************
(XEN) 3... 2... 1...
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen)
(XEN) Freed 280kB init memory.
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER4
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER8
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER12
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER16
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER20
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER24
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER28
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER32
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER36
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER40
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER44
(XEN) d0v0: vGICD: unhandled word write 0xffffffff to ICACTIVER0
[ 0.000000] Booting Linux on physical CPU 0x0
[ 0.000000] Linux version 4.14.0-rc7-linaro-hikey960+ () (gcc version 6.3.0 20170516 (Debian 6.3.0-18)) #8 SMP PREEMPT Mon Nov 5 19:57:52 NZDT 2018
[ 0.000000] Boot CPU: AArch64 Processor [410fd034]
[ 0.000000] Machine model: HiKey960
[ 0.000000] Xen 4.12 support found
[ 0.000000] efi: Getting EFI parameters from FDT:
[ 0.000000] efi: UEFI not found.
[ 0.000000] cma: Reserved 16 MiB at 0x00000000df000000
[ 0.000000] NUMA: No NUMA configuration found
[ 0.000000] NUMA: Faking a node at [mem 0x0000000000000000-0x00000000dfffffff]
[ 0.000000] NUMA: NODE_DATA [mem 0xdefb6800-0xdefb82ff]
[ 0.000000] Zone ranges:
[ 0.000000] DMA [mem 0x00000000c0000000-0x00000000dfffffff]
[ 0.000000] Normal empty
[ 0.000000] Movable zone start for each node
[ 0.000000] Early memory node ranges
[ 0.000000] node 0: [mem 0x00000000c0000000-0x00000000dfffffff]
[ 0.000000] Initmem setup node 0 [mem 0x00000000c0000000-0x00000000dfffffff]
[ 0.000000] psci: probing for conduit method from DT.
[ 0.000000] psci: PSCIv1.1 detected in firmware.
[ 0.000000] psci: Using standard PSCI v0.2 function IDs
[ 0.000000] psci: Trusted OS migration not required
[ 0.000000] percpu: Embedded 23 pages/cpu @ffff80001ef9c000 s55448 r8192 d30568 u94208
[ 0.000000] Detected VIPT I-cache on CPU0
[ 0.000000] CPU features: enabling workaround for ARM erratum 845719
[ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 129024
[ 0.000000] Policy zone: DMA
[ 0.000000] Kernel command line: console=tty0 console=hvc0 root=/dev/sdd10 rw efi=noruntime
[ 0.000000] PID hash table entries: 2048 (order: 2, 16384 bytes)
[ 0.000000] Memory: 473320K/524288K available (10236K kernel code, 1180K rwdata, 4684K rodata, 1152K init, 405K bss, 34584K reserved, 16384K cma-reserved)
[ 0.000000] Virtual kernel memory layout:
[ 0.000000] modules : 0xffff000000000000 - 0xffff000008000000 ( 128 MB)
[ 0.000000] vmalloc : 0xffff000008000000 - 0xffff7dffbfff0000 (129022 GB)
[ 0.000000] .text : 0xffff000008080000 - 0xffff000008a80000 ( 10240 KB)
[ 0.000000] .rodata : 0xffff000008a80000 - 0xffff000008f20000 ( 4736 KB)
[ 0.000000] .init : 0xffff000008f20000 - 0xffff000009040000 ( 1152 KB)
[ 0.000000] .data : 0xffff000009040000 - 0xffff000009167200 ( 1181 KB)
[ 0.000000] .bss : 0xffff000009167200 - 0xffff0000091cc960 ( 406 KB)
[ 0.000000] fixed : 0xffff7dfffe7fd000 - 0xffff7dfffec00000 ( 4108 KB)
[ 0.000000] PCI I/O : 0xffff7dfffee00000 - 0xffff7dffffe00000 ( 16 MB)
[ 0.000000] vmemmap : 0xffff7e0000000000 - 0xffff800000000000 ( 2048 GB maximum)
[ 0.000000] 0xffff7e0000000000 - 0xffff7e0000800000 ( 8 MB actual)
[ 0.000000] memory : 0xffff800000000000 - 0xffff800020000000 ( 512 MB)
[ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[ 0.000000] Preemptible hierarchical RCU implementation.
[ 0.000000] RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
[ 0.000000] Tasks RCU enabled.
[ 0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[ 0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[ 0.000000] arch_timer: cp15 timer(s) running at 1.92MHz (virt).
[ 0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0x1c570327e, max_idle_ns: 1763180809505 ns
[ 0.000004] sched_clock: 56 bits at 1920kHz, resolution 520ns, wraps every 4398046510977ns
[ 0.000105] clocksource: arm,sp804: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 58334160866034 ns
[ 0.000577] Console: colour dummy device 80x25
[ 0.000885] console [tty0] enabled
[ 0.342158] console [hvc0] enabled
[ 0.345707] Calibrating delay loop (skipped), value calculated using timer frequency.. 3.84 BogoMIPS (lpj=7680)
[ 0.355786] pid_max: default: 32768 minimum: 301
[ 0.360522] Security Framework initialized
[ 0.364807] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
[ 0.371747] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.378621] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.385288] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
[ 0.408528] ASID allocator initialised with 65536 entries
[ 0.414639] xen:grant_table: Grant tables using version 1 layout
[ 0.420588] Grant table initialized
[ 0.424137] xen:events: Using FIFO-based ABI
[ 0.428530] Xen: initializing cpu0
[ 0.436479] Hierarchical SRCU implementation.
[ 0.449785] EFI services will not be available.
[ 0.462263] smp: Bringing up secondary CPUs ...
[ 0.466703] smp: Brought up 1 node, 1 CPU
[ 0.470776] SMP: Total of 1 processors activated.
[ 0.475553] CPU features: detected feature: 32-bit EL0 Support
[ 0.481514] CPU: All CPU(s) started at EL1
[ 0.485620] alternatives: patching kernel code
[ 0.491125] devtmpfs: initialized
[ 0.499442] random: get_random_u32 called from bucket_table_alloc+0xf8/0x280 with crng_init=0
[ 0.512329] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[ 0.522020] futex hash table entries: 256 (order: 3, 32768 bytes)
[ 0.528755] pinctrl core: initialized pinctrl subsystem
[ 0.534123] random: fast init done
[ 0.539802] DMI not present or invalid.
[ 0.545483] NET: Registered protocol family 16
[ 0.554645] vdso: 2 pages (1 code @ ffff000008a86000, 1 data @ ffff000009045000)
[ 0.563385] hw-breakpoint: found 6 breakpoint and 4 watchpoint registers.
[ 0.570857] DMA: preallocated 256 KiB pool for atomic allocations
[ 0.577119] xen:swiotlb_xen: Warning: only able to allocate 4 MB for software IO TLB
[ 0.585539] software IO TLB [mem 0xdd000000-0xdd400000] (4MB) mapped at [ffff80001d000000-ffff80001d3fffff]
[ 0.596051] Serial: AMBA PL011 UART driver
[ 0.624374] hi3660-mbox e896b000.mailbox: Mailbox enabled
[ 0.644308] uart-pl011 ffd74000.serial: could not find pctldev for node /soc/pinmux@e896c000/uart3_pmx_func, deferring probe
[ 0.662034] uart-pl011 fdf01000.serial: could not find pctldev for node /soc/pinmux@e896c000/uart4_pmx_func, deferring probe
[ 0.941352] Synchronous External Abort: synchronous external abort (0x96000210) at 0xffff0000093fdfe0
[ 0.950601] Internal error: : 96000210 [#1] PREEMPT SMP
[ 0.955866] Modules linked in:
[ 0.958990] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 4.14.0-rc7-linaro-hikey960+ #8
[ 0.966791] Hardware name: HiKey960 (DT)
[ 0.970777] task: ffff80001d900000 task.stack: ffff000008058000
[ 0.976778] PC is at amba_device_try_add+0x108/0x260
[ 0.981791] LR is at amba_device_try_add+0xf0/0x260
[ 0.986735] pc : [<ffff0000084eada8>] lr : [<ffff0000084ead90>] pstate: 60000045
[ 0.994192] sp : ffff00000805bbf0
[ 0.997572] x29: ffff00000805bbf0 x28: 0000000000000000
[ 1.002953] x27: ffff0000090203a8 x26: 0000000000000000
[ 1.008326] x25: ffff80001dbf9810 x24: 0000000000000000
[ 1.013702] x23: ffff0000093fd000 x22: 0000000000001000
[ 1.019079] x21: ffff80001cc1b6f8 x20: 0000000000000000
[ 1.024455] x19: ffff80001cc1b400 x18: 0000000000000010
[ 1.029832] x17: 0000000000000001 x16: 00000000deadbeef
[ 1.035209] x15: 0000000000000006 x14: ffffffffffffffff
[ 1.040585] x13: 0000000000000020 x12: 0101010101010101
[ 1.045962] x11: 0000000000000020 x10: 0101010101010101
[ 1.051338] x9 : 0000000000000000 x8 : ffff80001cc0cf00
[ 1.056717] x7 : 0000000000000000 x6 : 000000000000003f
[ 1.062092] x5 : 0000000000000000 x4 : 0000000000000000
[ 1.067468] x3 : 0000000000000000 x2 : 0000000000000000
[ 1.072845] x1 : ffff80001d900000 x0 : ffff0000093fdfe0
[ 1.078223] Process swapper/0 (pid: 1, stack limit = 0xffff000008058000)
[ 1.084989] Call trace:
[ 1.087504] Exception stack(0xffff00000805bab0 to 0xffff00000805bbf0)
[ 1.094008] baa0: ffff0000093fdfe0 ffff80001d900000
[ 1.101902] bac0: 0000000000000000 0000000000000000 0000000000000000 0000000000000000
[ 1.109790] bae0: 000000000000003f 0000000000000000 ffff80001cc0cf00 0000000000000000
[ 1.117684] bb00: 0101010101010101 0000000000000020 0101010101010101 0000000000000020
[ 1.125576] bb20: ffffffffffffffff 0000000000000006 00000000deadbeef 0000000000000001
[ 1.133468] bb40: 0000000000000010 ffff80001cc1b400 0000000000000000 ffff80001cc1b6f8
[ 1.141356] bb60: 0000000000001000 ffff0000093fd000 0000000000000000 ffff80001dbf9810
[ 1.149248] bb80: 0000000000000000 ffff0000090203a8 0000000000000000 ffff00000805bbf0
[ 1.157139] bba0: ffff0000084ead90 ffff00000805bbf0 ffff0000084eada8 0000000060000045
[ 1.165034] bbc0: ffff00000805bbf0 ffff0000084ead90 ffffffffffffffff 00000000fffffffe
[ 1.172921] bbe0: ffff00000805bbf0 ffff0000084eada8
[ 1.177865] [<ffff0000084eada8>] amba_device_try_add+0x108/0x260
[ 1.183935] [<ffff0000084eafec>] amba_device_add+0x1c/0xd8
[ 1.189493] [<ffff00000890fbbc>] of_platform_bus_create+0x26c/0x300
[ 1.195814] [<ffff00000890fa74>] of_platform_bus_create+0x124/0x300
[ 1.202145] [<ffff00000890fd7c>] of_platform_populate+0x4c/0xb0
[ 1.208135] [<ffff000008f6e2ac>] of_platform_default_populate_init+0x64/0x78
[ 1.215247] [<ffff000008083978>] do_one_initcall+0x38/0x120
[ 1.220882] [<ffff000008f20d18>] kernel_init_freeable+0x184/0x224
[ 1.227038] [<ffff000008a6a360>] kernel_init+0x10/0x100
[ 1.232323] [<ffff000008084b60>] ret_from_fork+0x10/0x18
[ 1.237703] Code: d10082c0 52800002 8b0002e0 52800018 (b9400001)
[ 1.243880] ---[ end trace dcbf70aa30c979a8 ]---
[ 1.248573] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 1.248573]
[ 1.257826] ---[ end Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 1.257826]
[-- Attachment #3: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: Problems building and running Xen on Hikey960
2018-11-08 7:07 Problems building and running Xen on Hikey960 Matthew Daley
@ 2018-11-09 11:22 ` Julien Grall
2018-11-09 16:55 ` Omkar Bolla
2018-11-11 1:15 ` Matthew Daley
0 siblings, 2 replies; 13+ messages in thread
From: Julien Grall @ 2018-11-09 11:22 UTC (permalink / raw)
To: Matthew Daley, Xen-devel, Konrad Rzeszutek Wilk,
Stefano Stabellini, Omkar Bolla
On 08/11/2018 07:07, Matthew Daley wrote:
> Hi all,
Hi Matthew,
>
> I'm trying to build and run Linux (Debian) on top of Xen on a Hikey960
> ARM board but am having trouble.
>
> I've been following Konrad's guide on the Wiki (at
> https://wiki.xenproject.org/wiki/HiKey960), namely:
> * I have upgraded the board's base firmware to support UEFI
> * I am building the hikey960-upstream-rebase branch of
> https://github.com/96boards-hikey/linux.git (with a patch from Linus's
> tree to fix a Hynix UFS bug)
> * I am building the master branch of Xen unstable
> * I am building the master branch of GRUB (with a patch to fix a DTB
> padding bug)
>
> However, I see two problems when booting.
>
> Firstly, Xen fails to bring up any other CPUs but the one it is booting on:
>
> (XEN) Bringing up CPU1
> (XEN) Failed to bring up CPU1
> (XEN) Failed to bring up CPU 1 (error -9)
> (XEN) Bringing up CPU2
> (XEN) Failed to bring up CPU2
> (XEN) Failed to bring up CPU 2 (error -9)
> (XEN) Bringing up CPU3
> (XEN) Failed to bring up CPU3
> (XEN) Failed to bring up CPU 3 (error -9)
> (XEN) Bringing up CPU4
> (XEN) Failed to bring up CPU4
> (XEN) Failed to bring up CPU 4 (error -9)
> (XEN) Bringing up CPU5
> (XEN) Failed to bring up CPU5
> (XEN) Failed to bring up CPU 5 (error -9)
> (XEN) Bringing up CPU6
> (XEN) Failed to bring up CPU6
> (XEN) Failed to bring up CPU 6 (error -9)
> (XEN) Bringing up CPU7
> (XEN) Failed to bring up CPU7
> (XEN) Failed to bring up CPU 7 (error -9)
> (XEN) Brought up 1 CPUs
>
> I have traced this error code -9 being returned by call_psci_cpu_on.
A similar error was reported a couple of months on the mailing list. From the
report, a regression was introduced between Xen 4.8 and unstable.
Unfortunately, I don't have an hikey board to bisect it. May I ask if you can
bisect it? If you can point the offending commit, I should be able to provide
ideas why it breaks.
>
> Secondly, Linux fails when it tries to initialise AMBA devices:
>
> [ 0.941352] Synchronous External Abort: synchronous external abort
> (0x96000210) at 0xffff0000093fdfe
> 0
> [ 0.950601] Internal error: : 96000210 [#1] PREEMPT SMP
> [ 0.955866] Modules linked in:
> [ 0.958990] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> 4.14.0-rc7-linaro-hikey960+ #8
> [ 0.966791] Hardware name: HiKey960 (DT)
> [ 0.970777] task: ffff80001d900000 task.stack: ffff000008058000
> [ 0.976778] PC is at amba_device_try_add+0x108/0x260
> [ 0.981791] LR is at amba_device_try_add+0xf0/0x260
> [ 0.986735] pc : [<ffff0000084eada8>] lr : [<ffff0000084ead90>]
> pstate: 60000045
> [ 0.994192] sp : ffff00000805bbf0
> [ 0.997572] x29: ffff00000805bbf0 x28: 0000000000000000
> [ 1.002953] x27: ffff0000090203a8 x26: 0000000000000000
> [ 1.008326] x25: ffff80001dbf9810 x24: 0000000000000000
> [ 1.013702] x23: ffff0000093fd000 x22: 0000000000001000
> [ 1.019079] x21: ffff80001cc1b6f8 x20: 0000000000000000
> [ 1.024455] x19: ffff80001cc1b400 x18: 0000000000000010
> [ 1.029832] x17: 0000000000000001 x16: 00000000deadbeef
> [ 1.035209] x15: 0000000000000006 x14: ffffffffffffffff
> [ 1.040585] x13: 0000000000000020 x12: 0101010101010101
> [ 1.045962] x11: 0000000000000020 x10: 0101010101010101
> [ 1.051338] x9 : 0000000000000000 x8 : ffff80001cc0cf00
> [ 1.056717] x7 : 0000000000000000 x6 : 000000000000003f
> [ 1.062092] x5 : 0000000000000000 x4 : 0000000000000000
> [ 1.067468] x3 : 0000000000000000 x2 : 0000000000000000
> [ 1.072845] x1 : ffff80001d900000 x0 : ffff0000093fdfe0
> [ 1.078223] Process swapper/0 (pid: 1, stack limit = 0xffff000008058000)
> [ 1.084989] Call trace:
> [ 1.087504] Exception stack(0xffff00000805bab0 to 0xffff00000805bbf0)
> [ 1.094008] baa0:
> ffff0000093fdfe0 ffff80001d900000
> [ 1.101902] bac0: 0000000000000000 0000000000000000
> 0000000000000000 0000000000000000
> [ 1.109790] bae0: 000000000000003f 0000000000000000
> ffff80001cc0cf00 0000000000000000
> [ 1.117684] bb00: 0101010101010101 0000000000000020
> 0101010101010101 0000000000000020
> [ 1.125576] bb20: ffffffffffffffff 0000000000000006
> 00000000deadbeef 0000000000000001
> [ 1.133468] bb40: 0000000000000010 ffff80001cc1b400
> 0000000000000000 ffff80001cc1b6f8
> [ 1.141356] bb60: 0000000000001000 ffff0000093fd000
> 0000000000000000 ffff80001dbf9810
> [ 1.149248] bb80: 0000000000000000 ffff0000090203a8
> 0000000000000000 ffff00000805bbf0
> [ 1.157139] bba0: ffff0000084ead90 ffff00000805bbf0
> ffff0000084eada8 0000000060000045
> [ 1.165034] bbc0: ffff00000805bbf0 ffff0000084ead90
> ffffffffffffffff 00000000fffffffe
> [ 1.172921] bbe0: ffff00000805bbf0 ffff0000084eada8
> [ 1.177865] [<ffff0000084eada8>] amba_device_try_add+0x108/0x260
> [ 1.183935] [<ffff0000084eafec>] amba_device_add+0x1c/0xd8
> [ 1.189493] [<ffff00000890fbbc>] of_platform_bus_create+0x26c/0x300
> [ 1.195814] [<ffff00000890fa74>] of_platform_bus_create+0x124/0x300
> [ 1.202145] [<ffff00000890fd7c>] of_platform_populate+0x4c/0xb0
> [ 1.208135] [<ffff000008f6e2ac>] of_platform_default_populate_init+0x64/0x78
> [ 1.215247] [<ffff000008083978>] do_one_initcall+0x38/0x120
> [ 1.220882] [<ffff000008f20d18>] kernel_init_freeable+0x184/0x224
> [ 1.227038] [<ffff000008a6a360>] kernel_init+0x10/0x100
> [ 1.232323] [<ffff000008084b60>] ret_from_fork+0x10/0x18
> [ 1.237703] Code: d10082c0 52800002 8b0002e0 52800018 (b9400001)
> [ 1.243880] ---[ end trace dcbf70aa30c979a8 ]---
> [ 1.248573] Kernel panic - not syncing: Attempted to kill init!
> exitcode=0x0000000b
>
> This does not occur when booting the same Linux kernel without Xen.
>
> I have traced this to amba_device_try_add trying to access the pid and
> cid through ioremap'd addresses of this node in the device tree (in
> arch/arm64/boot/dts/hisilicon/hi3660-coresight.dtsi):
Thank you for confirming it works on Linux baremetal and tracked down the problem.
It is not entirely clear why you receive an external abort here. This may be due
to misconfiguration of the stage-2 page-tables.
Could you try Xen with this small changes? The patch should print a message is
the virtual abort was received by Xen but forwarded to the guest.
diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
index 51d2e42c77..f95135d030 100644
--- a/xen/arch/arm/traps.c
+++ b/xen/arch/arm/traps.c
@@ -660,6 +660,8 @@ static void inject_vabt_exception(struct cpu_user_regs *regs)
{
const union hsr hsr = { .bits = regs->hsr };
+ printk("%pv: Inject Virtual Abort\n", current);
+ dump_execution_state();
+
/*
* SVC/HVC/SMC already have an adjusted PC (See ARM ARM DDI 0487A.j
* D1.10.1 for more details), which we need to correct in order to
>
> /* A73 cluster internal coresight */
> etm@4,ed440000 {
> compatible = "arm,coresight-etm4x","arm,primecell";
> reg = <0 0xed440000 0 0x1000>;
> clocks = <&pclk>;
> clock-names = "apb_pclk";
> cpu = <&cpu4>;
> port {
> etm4_out_port: endpoint {
> remote-endpoint = <&funnel1_in_port0>;
> };
> };
> };
>
> ARM is still relatively new to me and I'm stuck what I should be
> attempting next. I would simply not compile Linux AMBA support (by not
> setting CONFIG_AMBA) but it appears that this is selected as a reverse
> dependency by CONFIG_ARM on Linux, so am unsure if this is wise or
> even possible.
I would just drop the node you copied above from the Device-Tree and see if you
can go further in the boot.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: Problems building and running Xen on Hikey960
2018-11-09 11:22 ` Julien Grall
@ 2018-11-09 16:55 ` Omkar Bolla
2018-11-09 17:29 ` Julien Grall
2018-11-11 1:15 ` Matthew Daley
1 sibling, 1 reply; 13+ messages in thread
From: Omkar Bolla @ 2018-11-09 16:55 UTC (permalink / raw)
To: julien.grall; +Cc: mattd, sstabellini, konrad.wilk, xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 9164 bytes --]
HI Mathues,
I checked your log and crashes in xen and linux. Do following things to fix
or move further.
On Fri, Nov 9, 2018 at 4:52 PM Julien Grall <julien.grall@arm.com> wrote:
>
>
> On 08/11/2018 07:07, Matthew Daley wrote:
> > Hi all,
>
> Hi Matthew,
>
> >
> > I'm trying to build and run Linux (Debian) on top of Xen on a Hikey960
> > ARM board but am having trouble.
> >
> > I've been following Konrad's guide on the Wiki (at
> > https://wiki.xenproject.org/wiki/HiKey960), namely:
> > * I have upgraded the board's base firmware to support UEFI
> > * I am building the hikey960-upstream-rebase branch of
> > https://github.com/96boards-hikey/linux.git (with a patch from Linus's
> > tree to fix a Hynix UFS bug)
> > * I am building the master branch of Xen unstable
> > * I am building the master branch of GRUB (with a patch to fix a DTB
> > padding bug)
> >
> > However, I see two problems when booting.
> >
> > Firstly, Xen fails to bring up any other CPUs but the one it is booting
> on:
> >
> > (XEN) Bringing up CPU1
> > (XEN) Failed to bring up CPU1
> > (XEN) Failed to bring up CPU 1 (error -9)
> > (XEN) Bringing up CPU2
> > (XEN) Failed to bring up CPU2
> > (XEN) Failed to bring up CPU 2 (error -9)
> > (XEN) Bringing up CPU3
> > (XEN) Failed to bring up CPU3
> > (XEN) Failed to bring up CPU 3 (error -9)
> > (XEN) Bringing up CPU4
> > (XEN) Failed to bring up CPU4
> > (XEN) Failed to bring up CPU 4 (error -9)
> > (XEN) Bringing up CPU5
> > (XEN) Failed to bring up CPU5
> > (XEN) Failed to bring up CPU 5 (error -9)
> > (XEN) Bringing up CPU6
> > (XEN) Failed to bring up CPU6
> > (XEN) Failed to bring up CPU 6 (error -9)
> > (XEN) Bringing up CPU7
> > (XEN) Failed to bring up CPU7
> > (XEN) Failed to bring up CPU 7 (error -9)
> > (XEN) Brought up 1 CPUs
> >
> > I have traced this error code -9 being returned by call_psci_cpu_on.
>
> A similar error was reported a couple of months on the mailing list. From
> the
> report, a regression was introduced between Xen 4.8 and unstable.
>
> Unfortunately, I don't have an hikey board to bisect it. May I ask if you
> can
> bisect it? If you can point the offending commit, I should be able to
> provide
> ideas why it breaks.
>
> As Julien said,
For your CPUs problem in Hikey960, use 4.8 of xen source, Then all CPUs
will be enabled and worked properly.
> >
> > Secondly, Linux fails when it tries to initialise AMBA devices:
> >
> > [ 0.941352] Synchronous External Abort: synchronous external abort
> > (0x96000210) at 0xffff0000093fdfe
> > 0
> > [ 0.950601] Internal error: : 96000210 [#1] PREEMPT SMP
> > [ 0.955866] Modules linked in:
> > [ 0.958990] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> > 4.14.0-rc7-linaro-hikey960+ #8
> > [ 0.966791] Hardware name: HiKey960 (DT)
> > [ 0.970777] task: ffff80001d900000 task.stack: ffff000008058000
> > [ 0.976778] PC is at amba_device_try_add+0x108/0x260
> > [ 0.981791] LR is at amba_device_try_add+0xf0/0x260
> > [ 0.986735] pc : [<ffff0000084eada8>] lr : [<ffff0000084ead90>]
> > pstate: 60000045
> > [ 0.994192] sp : ffff00000805bbf0
> > [ 0.997572] x29: ffff00000805bbf0 x28: 0000000000000000
> > [ 1.002953] x27: ffff0000090203a8 x26: 0000000000000000
> > [ 1.008326] x25: ffff80001dbf9810 x24: 0000000000000000
> > [ 1.013702] x23: ffff0000093fd000 x22: 0000000000001000
> > [ 1.019079] x21: ffff80001cc1b6f8 x20: 0000000000000000
> > [ 1.024455] x19: ffff80001cc1b400 x18: 0000000000000010
> > [ 1.029832] x17: 0000000000000001 x16: 00000000deadbeef
> > [ 1.035209] x15: 0000000000000006 x14: ffffffffffffffff
> > [ 1.040585] x13: 0000000000000020 x12: 0101010101010101
> > [ 1.045962] x11: 0000000000000020 x10: 0101010101010101
> > [ 1.051338] x9 : 0000000000000000 x8 : ffff80001cc0cf00
> > [ 1.056717] x7 : 0000000000000000 x6 : 000000000000003f
> > [ 1.062092] x5 : 0000000000000000 x4 : 0000000000000000
> > [ 1.067468] x3 : 0000000000000000 x2 : 0000000000000000
> > [ 1.072845] x1 : ffff80001d900000 x0 : ffff0000093fdfe0
> > [ 1.078223] Process swapper/0 (pid: 1, stack limit =
> 0xffff000008058000)
> > [ 1.084989] Call trace:
> > [ 1.087504] Exception stack(0xffff00000805bab0 to 0xffff00000805bbf0)
> > [ 1.094008] baa0:
> > ffff0000093fdfe0 ffff80001d900000
> > [ 1.101902] bac0: 0000000000000000 0000000000000000
> > 0000000000000000 0000000000000000
> > [ 1.109790] bae0: 000000000000003f 0000000000000000
> > ffff80001cc0cf00 0000000000000000
> > [ 1.117684] bb00: 0101010101010101 0000000000000020
> > 0101010101010101 0000000000000020
> > [ 1.125576] bb20: ffffffffffffffff 0000000000000006
> > 00000000deadbeef 0000000000000001
> > [ 1.133468] bb40: 0000000000000010 ffff80001cc1b400
> > 0000000000000000 ffff80001cc1b6f8
> > [ 1.141356] bb60: 0000000000001000 ffff0000093fd000
> > 0000000000000000 ffff80001dbf9810
> > [ 1.149248] bb80: 0000000000000000 ffff0000090203a8
> > 0000000000000000 ffff00000805bbf0
> > [ 1.157139] bba0: ffff0000084ead90 ffff00000805bbf0
> > ffff0000084eada8 0000000060000045
> > [ 1.165034] bbc0: ffff00000805bbf0 ffff0000084ead90
> > ffffffffffffffff 00000000fffffffe
> > [ 1.172921] bbe0: ffff00000805bbf0 ffff0000084eada8
> > [ 1.177865] [<ffff0000084eada8>] amba_device_try_add+0x108/0x260
> > [ 1.183935] [<ffff0000084eafec>] amba_device_add+0x1c/0xd8
> > [ 1.189493] [<ffff00000890fbbc>] of_platform_bus_create+0x26c/0x300
> > [ 1.195814] [<ffff00000890fa74>] of_platform_bus_create+0x124/0x300
> > [ 1.202145] [<ffff00000890fd7c>] of_platform_populate+0x4c/0xb0
> > [ 1.208135] [<ffff000008f6e2ac>]
> of_platform_default_populate_init+0x64/0x78
> > [ 1.215247] [<ffff000008083978>] do_one_initcall+0x38/0x120
> > [ 1.220882] [<ffff000008f20d18>] kernel_init_freeable+0x184/0x224
> > [ 1.227038] [<ffff000008a6a360>] kernel_init+0x10/0x100
> > [ 1.232323] [<ffff000008084b60>] ret_from_fork+0x10/0x18
> > [ 1.237703] Code: d10082c0 52800002 8b0002e0 52800018 (b9400001)
> > [ 1.243880] ---[ end trace dcbf70aa30c979a8 ]---
> > [ 1.248573] Kernel panic - not syncing: Attempted to kill init!
> > exitcode=0x0000000b
> >
> > This does not occur when booting the same Linux kernel without Xen.
> >
> > I have traced this to amba_device_try_add trying to access the pid and
> > cid through ioremap'd addresses of this node in the device tree (in
> > arch/arm64/boot/dts/hisilicon/hi3660-coresight.dtsi):
>
> Thank you for confirming it works on Linux baremetal and tracked down the
> problem.
>
> It is not entirely clear why you receive an external abort here. This may
> be due
> to misconfiguration of the stage-2 page-tables.
>
> Could you try Xen with this small changes? The patch should print a
> message is
> the virtual abort was received by Xen but forwarded to the guest.
>
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 51d2e42c77..f95135d030 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -660,6 +660,8 @@ static void inject_vabt_exception(struct cpu_user_regs
> *regs)
> {
> const union hsr hsr = { .bits = regs->hsr };
>
> + printk("%pv: Inject Virtual Abort\n", current);
> + dump_execution_state();
> +
> /*
> * SVC/HVC/SMC already have an adjusted PC (See ARM ARM DDI 0487A.j
> * D1.10.1 for more details), which we need to correct in order to
>
> >
> > /* A73 cluster internal coresight */
> > etm@4,ed440000 {
> > compatible = "arm,coresight-etm4x","arm,primecell";
> > reg = <0 0xed440000 0 0x1000>;
> > clocks = <&pclk>;
> > clock-names = "apb_pclk";
> > cpu = <&cpu4>;
> > port {
> > etm4_out_port: endpoint {
> > remote-endpoint = <&funnel1_in_port0>;
> > };
> > };
> > };
> >
> > ARM is still relatively new to me and I'm stuck what I should be
> > attempting next. I would simply not compile Linux AMBA support (by not
> > setting CONFIG_AMBA) but it appears that this is selected as a reverse
> > dependency by CONFIG_ARM on Linux, so am unsure if this is wise or
> > even possible.
>
> I would just drop the node you copied above from the Device-Tree and see
> if you
> can go further in the boot.
>
>
As Julien said,
Disable all 'etm' nodes in device-tree for all CPUs. This will fix all amba
CPU devices linux crashes.
Cheers,
>
> --
> Julien Grall
>
Thanks,
Omkar B
--
This
message contains confidential information and is intended only
for the
individual(s) named. If you are not the intended
recipient, you are
notified that disclosing, copying, distributing or taking any
action in
reliance on the contents of this mail and attached file/s is strictly
prohibited. Please notify the
sender immediately and delete this e-mail
from your system. E-mail transmission
cannot be guaranteed to be secured or
error-free as information could be
intercepted, corrupted, lost, destroyed,
arrive late or incomplete, or contain
viruses. The sender therefore does
not accept liability for any errors or
omissions in the contents of this
message, which arise as a result of e-mail
transmission.
[-- Attachment #1.2: Type: text/html, Size: 12444 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: Problems building and running Xen on Hikey960
2018-11-09 16:55 ` Omkar Bolla
@ 2018-11-09 17:29 ` Julien Grall
0 siblings, 0 replies; 13+ messages in thread
From: Julien Grall @ 2018-11-09 17:29 UTC (permalink / raw)
To: Omkar Bolla; +Cc: mattd, sstabellini, konrad.wilk, xen-devel
Hi Omkar,
Please configure your e-mail client to quote properly and remove the disclaimer
below.
On 09/11/2018 16:55, Omkar Bolla wrote:
> On Fri, Nov 9, 2018 at 4:52 PM Julien Grall <julien.grall@arm.com
> <mailto:julien.grall@arm.com>> wrote:
>
>
>
> On 08/11/2018 07:07, Matthew Daley wrote:
> > Hi all,
>
> Hi Matthew,
>
> >
> > I'm trying to build and run Linux (Debian) on top of Xen on a Hikey960
> > ARM board but am having trouble.
> >
> > I've been following Konrad's guide on the Wiki (at
> > https://wiki.xenproject.org/wiki/HiKey960), namely:
> > * I have upgraded the board's base firmware to support UEFI
> > * I am building the hikey960-upstream-rebase branch of
> > https://github.com/96boards-hikey/linux.git (with a patch from Linus's
> > tree to fix a Hynix UFS bug)
> > * I am building the master branch of Xen unstable
> > * I am building the master branch of GRUB (with a patch to fix a DTB
> > padding bug)
> >
> > However, I see two problems when booting.
> >
> > Firstly, Xen fails to bring up any other CPUs but the one it is booting on:
> >
> > (XEN) Bringing up CPU1
> > (XEN) Failed to bring up CPU1
> > (XEN) Failed to bring up CPU 1 (error -9)
> > (XEN) Bringing up CPU2
> > (XEN) Failed to bring up CPU2
> > (XEN) Failed to bring up CPU 2 (error -9)
> > (XEN) Bringing up CPU3
> > (XEN) Failed to bring up CPU3
> > (XEN) Failed to bring up CPU 3 (error -9)
> > (XEN) Bringing up CPU4
> > (XEN) Failed to bring up CPU4
> > (XEN) Failed to bring up CPU 4 (error -9)
> > (XEN) Bringing up CPU5
> > (XEN) Failed to bring up CPU5
> > (XEN) Failed to bring up CPU 5 (error -9)
> > (XEN) Bringing up CPU6
> > (XEN) Failed to bring up CPU6
> > (XEN) Failed to bring up CPU 6 (error -9)
> > (XEN) Bringing up CPU7
> > (XEN) Failed to bring up CPU7
> > (XEN) Failed to bring up CPU 7 (error -9)
> > (XEN) Brought up 1 CPUs
> >
> > I have traced this error code -9 being returned by call_psci_cpu_on.
>
> A similar error was reported a couple of months on the mailing list. From the
> report, a regression was introduced between Xen 4.8 and unstable.
>
> Unfortunately, I don't have an hikey board to bisect it. May I ask if you can
> bisect it? If you can point the offending commit, I should be able to provide
> ideas why it breaks.
>
> As Julien said,
> For your CPUs problem in Hikey960, use 4.8 of xen source, Then all CPUs will be
> enabled and worked properly.
I would strongly discourage anyone to use Xen 4.8 for the Hikey 960. Firstly
because this release is out of support. Secondly because it does not contain any
fixes for Spectre on Arm.
The Hikey 960 has cores vulnerable to some variants of Spectre & Meltdown. So an
upgrade to the latest Xen is highly recommended to benefits of the latest
security fixes.
For the benefit of everyone it would be nice if someone actually spend a couple
of hours bisecting Xen on Hikey to find the offending commit. I can provide
support for that.
Cheers,
[1]
https://developer.arm.com/support/arm-security-updates/speculative-processor-vulnerability
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Problems building and running Xen on Hikey960
2018-11-09 11:22 ` Julien Grall
2018-11-09 16:55 ` Omkar Bolla
@ 2018-11-11 1:15 ` Matthew Daley
2018-11-12 13:01 ` Julien Grall
1 sibling, 1 reply; 13+ messages in thread
From: Matthew Daley @ 2018-11-11 1:15 UTC (permalink / raw)
To: julien.grall
Cc: omkar.bolla, Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
Hi Julien,
On Sat, 10 Nov 2018 at 00:22, Julien Grall <julien.grall@arm.com> wrote:
> > Firstly, Xen fails to bring up any other CPUs but the one it is booting on:
> >
> > (XEN) Bringing up CPU1
> > (XEN) Failed to bring up CPU1
> > (XEN) Failed to bring up CPU 1 (error -9)
> > (XEN) Bringing up CPU2
> > (XEN) Failed to bring up CPU2
> > (XEN) Failed to bring up CPU 2 (error -9)
> > (XEN) Bringing up CPU3
> > (XEN) Failed to bring up CPU3
> > (XEN) Failed to bring up CPU 3 (error -9)
> > (XEN) Bringing up CPU4
> > (XEN) Failed to bring up CPU4
> > (XEN) Failed to bring up CPU 4 (error -9)
> > (XEN) Bringing up CPU5
> > (XEN) Failed to bring up CPU5
> > (XEN) Failed to bring up CPU 5 (error -9)
> > (XEN) Bringing up CPU6
> > (XEN) Failed to bring up CPU6
> > (XEN) Failed to bring up CPU 6 (error -9)
> > (XEN) Bringing up CPU7
> > (XEN) Failed to bring up CPU7
> > (XEN) Failed to bring up CPU 7 (error -9)
> > (XEN) Brought up 1 CPUs
> >
> > I have traced this error code -9 being returned by call_psci_cpu_on.
>
> A similar error was reported a couple of months on the mailing list. From the
> report, a regression was introduced between Xen 4.8 and unstable.
>
> Unfortunately, I don't have an hikey board to bisect it. May I ask if you can
> bisect it? If you can point the offending commit, I should be able to provide
> ideas why it breaks.
I managed to bisect this to commit
9f954a5e90414d10632e6c2fef5a33ea8a4a1e4e. Reverting this revert (!) on
top of current master leads to the CPUs (at least the big cores, as
expected) being brought online correctly:
(XEN) Bringing up CPU1
(XEN) CPU 1 booted.
(XEN) Bringing up CPU2
(XEN) CPU 2 booted.
(XEN) Bringing up CPU3
(XEN) CPU 3 booted.
(XEN) Bringing up CPU4
(XEN) CPU4 MIDR (0x410fd091) does not match boot CPU MIDR (0x410fd034),
(XEN) disable cpu (see big.LITTLE.txt under docs/).
(XEN) CPU4 never came online
(XEN) Failed to bring up CPU 4 (error -5)
(XEN) Bringing up CPU5
(XEN) CPU5 MIDR (0x410fd091) does not match boot CPU MIDR (0x410fd034),
(XEN) disable cpu (see big.LITTLE.txt under docs/).
(XEN) CPU5 never came online
(XEN) Failed to bring up CPU 5 (error -5)
(XEN) Bringing up CPU6
(XEN) CPU6 MIDR (0x410fd091) does not match boot CPU MIDR (0x410fd034),
(XEN) disable cpu (see big.LITTLE.txt under docs/).
(XEN) CPU6 never came online
(XEN) Failed to bring up CPU 6 (error -5)
(XEN) Bringing up CPU7
(XEN) CPU7 MIDR (0x410fd091) does not match boot CPU MIDR (0x410fd034),
(XEN) disable cpu (see big.LITTLE.txt under docs/).
(XEN) CPU7 never came online
(XEN) Failed to bring up CPU 7 (error -5)
(XEN) Brought up 4 CPUs
> > Secondly, Linux fails when it tries to initialise AMBA devices:
> >
> > [ 0.941352] Synchronous External Abort: synchronous external abort
> > (0x96000210) at 0xffff0000093fdfe
> > 0
> > [ 0.950601] Internal error: : 96000210 [#1] PREEMPT SMP
> > [ 0.955866] Modules linked in:
> > [ 0.958990] CPU: 0 PID: 1 Comm: swapper/0 Not tainted
> > 4.14.0-rc7-linaro-hikey960+ #8
> > [ 0.966791] Hardware name: HiKey960 (DT)
> > [ 0.970777] task: ffff80001d900000 task.stack: ffff000008058000
> > [ 0.976778] PC is at amba_device_try_add+0x108/0x260
> > [ 0.981791] LR is at amba_device_try_add+0xf0/0x260
> > [ 0.986735] pc : [<ffff0000084eada8>] lr : [<ffff0000084ead90>]
> > pstate: 60000045
> > [ 0.994192] sp : ffff00000805bbf0
> > [ 0.997572] x29: ffff00000805bbf0 x28: 0000000000000000
> > [ 1.002953] x27: ffff0000090203a8 x26: 0000000000000000
> > [ 1.008326] x25: ffff80001dbf9810 x24: 0000000000000000
> > [ 1.013702] x23: ffff0000093fd000 x22: 0000000000001000
> > [ 1.019079] x21: ffff80001cc1b6f8 x20: 0000000000000000
> > [ 1.024455] x19: ffff80001cc1b400 x18: 0000000000000010
> > [ 1.029832] x17: 0000000000000001 x16: 00000000deadbeef
> > [ 1.035209] x15: 0000000000000006 x14: ffffffffffffffff
> > [ 1.040585] x13: 0000000000000020 x12: 0101010101010101
> > [ 1.045962] x11: 0000000000000020 x10: 0101010101010101
> > [ 1.051338] x9 : 0000000000000000 x8 : ffff80001cc0cf00
> > [ 1.056717] x7 : 0000000000000000 x6 : 000000000000003f
> > [ 1.062092] x5 : 0000000000000000 x4 : 0000000000000000
> > [ 1.067468] x3 : 0000000000000000 x2 : 0000000000000000
> > [ 1.072845] x1 : ffff80001d900000 x0 : ffff0000093fdfe0
> > [ 1.078223] Process swapper/0 (pid: 1, stack limit = 0xffff000008058000)
> > [ 1.084989] Call trace:
> > [ 1.087504] Exception stack(0xffff00000805bab0 to 0xffff00000805bbf0)
> > [ 1.094008] baa0:
> > ffff0000093fdfe0 ffff80001d900000
> > [ 1.101902] bac0: 0000000000000000 0000000000000000
> > 0000000000000000 0000000000000000
> > [ 1.109790] bae0: 000000000000003f 0000000000000000
> > ffff80001cc0cf00 0000000000000000
> > [ 1.117684] bb00: 0101010101010101 0000000000000020
> > 0101010101010101 0000000000000020
> > [ 1.125576] bb20: ffffffffffffffff 0000000000000006
> > 00000000deadbeef 0000000000000001
> > [ 1.133468] bb40: 0000000000000010 ffff80001cc1b400
> > 0000000000000000 ffff80001cc1b6f8
> > [ 1.141356] bb60: 0000000000001000 ffff0000093fd000
> > 0000000000000000 ffff80001dbf9810
> > [ 1.149248] bb80: 0000000000000000 ffff0000090203a8
> > 0000000000000000 ffff00000805bbf0
> > [ 1.157139] bba0: ffff0000084ead90 ffff00000805bbf0
> > ffff0000084eada8 0000000060000045
> > [ 1.165034] bbc0: ffff00000805bbf0 ffff0000084ead90
> > ffffffffffffffff 00000000fffffffe
> > [ 1.172921] bbe0: ffff00000805bbf0 ffff0000084eada8
> > [ 1.177865] [<ffff0000084eada8>] amba_device_try_add+0x108/0x260
> > [ 1.183935] [<ffff0000084eafec>] amba_device_add+0x1c/0xd8
> > [ 1.189493] [<ffff00000890fbbc>] of_platform_bus_create+0x26c/0x300
> > [ 1.195814] [<ffff00000890fa74>] of_platform_bus_create+0x124/0x300
> > [ 1.202145] [<ffff00000890fd7c>] of_platform_populate+0x4c/0xb0
> > [ 1.208135] [<ffff000008f6e2ac>] of_platform_default_populate_init+0x64/0x78
> > [ 1.215247] [<ffff000008083978>] do_one_initcall+0x38/0x120
> > [ 1.220882] [<ffff000008f20d18>] kernel_init_freeable+0x184/0x224
> > [ 1.227038] [<ffff000008a6a360>] kernel_init+0x10/0x100
> > [ 1.232323] [<ffff000008084b60>] ret_from_fork+0x10/0x18
> > [ 1.237703] Code: d10082c0 52800002 8b0002e0 52800018 (b9400001)
> > [ 1.243880] ---[ end trace dcbf70aa30c979a8 ]---
> > [ 1.248573] Kernel panic - not syncing: Attempted to kill init!
> > exitcode=0x0000000b
> >
> > This does not occur when booting the same Linux kernel without Xen.
> >
> > I have traced this to amba_device_try_add trying to access the pid and
> > cid through ioremap'd addresses of this node in the device tree (in
> > arch/arm64/boot/dts/hisilicon/hi3660-coresight.dtsi):
>
> Thank you for confirming it works on Linux baremetal and tracked down the problem.
>
> It is not entirely clear why you receive an external abort here. This may be due
> to misconfiguration of the stage-2 page-tables.
>
> Could you try Xen with this small changes? The patch should print a message is
> the virtual abort was received by Xen but forwarded to the guest.
>
> diff --git a/xen/arch/arm/traps.c b/xen/arch/arm/traps.c
> index 51d2e42c77..f95135d030 100644
> --- a/xen/arch/arm/traps.c
> +++ b/xen/arch/arm/traps.c
> @@ -660,6 +660,8 @@ static void inject_vabt_exception(struct cpu_user_regs *regs)
> {
> const union hsr hsr = { .bits = regs->hsr };
>
> + printk("%pv: Inject Virtual Abort\n", current);
> + dump_execution_state();
> +
> /*
> * SVC/HVC/SMC already have an adjusted PC (See ARM ARM DDI 0487A.j
> * D1.10.1 for more details), which we need to correct in order to
I did not try your patch, but...
> > /* A73 cluster internal coresight */
> > etm@4,ed440000 {
> > compatible = "arm,coresight-etm4x","arm,primecell";
> > reg = <0 0xed440000 0 0x1000>;
> > clocks = <&pclk>;
> > clock-names = "apb_pclk";
> > cpu = <&cpu4>;
> > port {
> > etm4_out_port: endpoint {
> > remote-endpoint = <&funnel1_in_port0>;
> > };
> > };
> > };
> >
> > ARM is still relatively new to me and I'm stuck what I should be
> > attempting next. I would simply not compile Linux AMBA support (by not
> > setting CONFIG_AMBA) but it appears that this is selected as a reverse
> > dependency by CONFIG_ARM on Linux, so am unsure if this is wise or
> > even possible.
>
> I would just drop the node you copied above from the Device-Tree and see if you
> can go further in the boot.
...turns out that these nodes appear to belong to the little cores
(which were not brought online previously and still aren't with the
reverted revert), so munging the DT so as to remove these nodes fixes
this problem too.
Thanks for your help!
- Matthew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: Problems building and running Xen on Hikey960
2018-11-11 1:15 ` Matthew Daley
@ 2018-11-12 13:01 ` Julien Grall
2018-11-13 22:43 ` Matthew Daley
0 siblings, 1 reply; 13+ messages in thread
From: Julien Grall @ 2018-11-12 13:01 UTC (permalink / raw)
To: Matthew Daley
Cc: omkar.bolla, Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
On 11/11/18 1:15 AM, Matthew Daley wrote:
> Hi Julien,
Hi Matthew,
> On Sat, 10 Nov 2018 at 00:22, Julien Grall <julien.grall@arm.com> wrote:
>>> Firstly, Xen fails to bring up any other CPUs but the one it is booting on:
>>>
>>> (XEN) Bringing up CPU1
>>> (XEN) Failed to bring up CPU1
>>> (XEN) Failed to bring up CPU 1 (error -9)
>>> (XEN) Bringing up CPU2
>>> (XEN) Failed to bring up CPU2
>>> (XEN) Failed to bring up CPU 2 (error -9)
>>> (XEN) Bringing up CPU3
>>> (XEN) Failed to bring up CPU3
>>> (XEN) Failed to bring up CPU 3 (error -9)
>>> (XEN) Bringing up CPU4
>>> (XEN) Failed to bring up CPU4
>>> (XEN) Failed to bring up CPU 4 (error -9)
>>> (XEN) Bringing up CPU5
>>> (XEN) Failed to bring up CPU5
>>> (XEN) Failed to bring up CPU 5 (error -9)
>>> (XEN) Bringing up CPU6
>>> (XEN) Failed to bring up CPU6
>>> (XEN) Failed to bring up CPU 6 (error -9)
>>> (XEN) Bringing up CPU7
>>> (XEN) Failed to bring up CPU7
>>> (XEN) Failed to bring up CPU 7 (error -9)
>>> (XEN) Brought up 1 CPUs
>>>
>>> I have traced this error code -9 being returned by call_psci_cpu_on.
>>
>> A similar error was reported a couple of months on the mailing list. From the
>> report, a regression was introduced between Xen 4.8 and unstable.
>>
>> Unfortunately, I don't have an hikey board to bisect it. May I ask if you can
>> bisect it? If you can point the offending commit, I should be able to provide
>> ideas why it breaks.
>
> I managed to bisect this to commit
> 9f954a5e90414d10632e6c2fef5a33ea8a4a1e4e. Reverting this revert (!) on
> top of current master leads to the CPUs (at least the big cores, as
> expected) being brought online correctly:
Thank you for bisecting Xen, it is actually poin to the commit I
suspected. I have been told that entry point for secondary CPUs on the
Hikey board may require to be below 4GB.
I am not entirely sure how to address this yet. Would you mind to have a
try at the following patch?
diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
index 80f00286d3..f1fdfa4c91 100644
--- a/xen/arch/arm/setup.c
+++ b/xen/arch/arm/setup.c
@@ -409,13 +409,11 @@ static paddr_t __init get_xen_paddr(void)
if ( !e )
continue;
-#ifdef CONFIG_ARM_32
/* Xen must be under 4GB */
if ( e > 0x100000000ULL )
e = 0x100000000ULL;
if ( e < bank->start )
continue;
-#endif
s = e - min_size;
[...]
> ...turns out that these nodes appear to belong to the little cores
> (which were not brought online previously and still aren't with the
> reverted revert), so munging the DT so as to remove these nodes fixes
> this problem too.
We disable support of big.LITTLE by default because Xen currently
assumes all the CPUs are the same. This is not the case in the
big.LITTLE case.
You can override that behavior by adding hmp-unsafe on Xen command line.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: Problems building and running Xen on Hikey960
2018-11-12 13:01 ` Julien Grall
@ 2018-11-13 22:43 ` Matthew Daley
2018-11-22 20:51 ` Julien Grall
0 siblings, 1 reply; 13+ messages in thread
From: Matthew Daley @ 2018-11-13 22:43 UTC (permalink / raw)
To: julien.grall; +Cc: Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
> On 11/11/18 1:15 AM, Matthew Daley wrote:
> > Hi Julien,
>
> Hi Matthew,
>
> > On Sat, 10 Nov 2018 at 00:22, Julien Grall <julien.grall@arm.com> wrote:
> >>> Firstly, Xen fails to bring up any other CPUs but the one it is booting on:
> >>>
> >>> (XEN) Bringing up CPU1
> >>> (XEN) Failed to bring up CPU1
> >>> (XEN) Failed to bring up CPU 1 (error -9)
> >>> (XEN) Bringing up CPU2
> >>> (XEN) Failed to bring up CPU2
> >>> (XEN) Failed to bring up CPU 2 (error -9)
> >>> (XEN) Bringing up CPU3
> >>> (XEN) Failed to bring up CPU3
> >>> (XEN) Failed to bring up CPU 3 (error -9)
> >>> (XEN) Bringing up CPU4
> >>> (XEN) Failed to bring up CPU4
> >>> (XEN) Failed to bring up CPU 4 (error -9)
> >>> (XEN) Bringing up CPU5
> >>> (XEN) Failed to bring up CPU5
> >>> (XEN) Failed to bring up CPU 5 (error -9)
> >>> (XEN) Bringing up CPU6
> >>> (XEN) Failed to bring up CPU6
> >>> (XEN) Failed to bring up CPU 6 (error -9)
> >>> (XEN) Bringing up CPU7
> >>> (XEN) Failed to bring up CPU7
> >>> (XEN) Failed to bring up CPU 7 (error -9)
> >>> (XEN) Brought up 1 CPUs
> >>>
> >>> I have traced this error code -9 being returned by call_psci_cpu_on.
> >>
> >> A similar error was reported a couple of months on the mailing list. From the
> >> report, a regression was introduced between Xen 4.8 and unstable.
> >>
> >> Unfortunately, I don't have an hikey board to bisect it. May I ask if you can
> >> bisect it? If you can point the offending commit, I should be able to provide
> >> ideas why it breaks.
> >
> > I managed to bisect this to commit
> > 9f954a5e90414d10632e6c2fef5a33ea8a4a1e4e. Reverting this revert (!) on
> > top of current master leads to the CPUs (at least the big cores, as
> > expected) being brought online correctly:
> Thank you for bisecting Xen, it is actually poin to the commit I
> suspected. I have been told that entry point for secondary CPUs on the
> Hikey board may require to be below 4GB.
>
> I am not entirely sure how to address this yet. Would you mind to have a
> try at the following patch?
>
> diff --git a/xen/arch/arm/setup.c b/xen/arch/arm/setup.c
> index 80f00286d3..f1fdfa4c91 100644
> --- a/xen/arch/arm/setup.c
> +++ b/xen/arch/arm/setup.c
> @@ -409,13 +409,11 @@ static paddr_t __init get_xen_paddr(void)
> if ( !e )
> continue;
>
> -#ifdef CONFIG_ARM_32
> /* Xen must be under 4GB */
> if ( e > 0x100000000ULL )
> e = 0x100000000ULL;
> if ( e < bank->start )
> continue;
> -#endif
>
> s = e - min_size;
>
I gave this a go but unfortunately the same problem occurs (error
-9s). Just to check nothing weird is happening I added a printk to
check the value of __pa(init_secondary) in call_psci_cpu_on, giving
0xdfe00180.
> > ...turns out that these nodes appear to belong to the little cores
> > (which were not brought online previously and still aren't with the
> > reverted revert), so munging the DT so as to remove these nodes fixes
> > this problem too.
>
> We disable support of big.LITTLE by default because Xen currently
> assumes all the CPUs are the same. This is not the case in the
> big.LITTLE case.
>
> You can override that behavior by adding hmp-unsafe on Xen command line.
Right.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Problems building and running Xen on Hikey960
2018-11-13 22:43 ` Matthew Daley
@ 2018-11-22 20:51 ` Julien Grall
2018-11-26 1:07 ` Matthew Daley
0 siblings, 1 reply; 13+ messages in thread
From: Julien Grall @ 2018-11-22 20:51 UTC (permalink / raw)
To: Matthew Daley; +Cc: Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
Hi Matthew,
Sorry for the late answer and thank you for testing the patch.
On 11/13/18 10:43 PM, Matthew Daley wrote:
> On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
>> On 11/11/18 1:15 AM, Matthew Daley wrote:
> I gave this a go but unfortunately the same problem occurs (error
> -9s). Just to check nothing weird is happening I added a printk to
> check the value of __pa(init_secondary) in call_psci_cpu_on, giving
> 0xdfe00180.
I have posted a patch that disables completely the relocation [1]. Could
you have a try and see whether it works now?
Cheers,
[1] https://lists.xen.org/archives/html/xen-devel/2018-11/msg02638.html
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Problems building and running Xen on Hikey960
2018-11-22 20:51 ` Julien Grall
@ 2018-11-26 1:07 ` Matthew Daley
2018-11-28 6:43 ` Matthew Daley
0 siblings, 1 reply; 13+ messages in thread
From: Matthew Daley @ 2018-11-26 1:07 UTC (permalink / raw)
To: julien.grall; +Cc: Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
[-- Attachment #1.1: Type: text/plain, Size: 903 bytes --]
On Fri, 23 Nov 2018 at 09:51, Julien Grall <julien.grall@arm.com> wrote:
> Hi Matthew,
>
> Sorry for the late answer and thank you for testing the patch.
>
> On 11/13/18 10:43 PM, Matthew Daley wrote:
> > On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
> >> On 11/11/18 1:15 AM, Matthew Daley wrote:
> > I gave this a go but unfortunately the same problem occurs (error
> > -9s). Just to check nothing weird is happening I added a printk to
> > check the value of __pa(init_secondary) in call_psci_cpu_on, giving
> > 0xdfe00180.
>
> I have posted a patch that disables completely the relocation [1]. Could
> you have a try and see whether it works now?
>
> Cheers,
>
> [1] https://lists.xen.org/archives/html/xen-devel/2018-11/msg02638.html
>
> --
> Julien Grall
>
Hi Julien,
Unfortunately with this patch it appears to freeze after the call
to switch_ttbr(ttbr).
- Matthew
[-- Attachment #1.2: Type: text/html, Size: 1567 bytes --]
[-- Attachment #2: Type: text/plain, Size: 157 bytes --]
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Problems building and running Xen on Hikey960
2018-11-26 1:07 ` Matthew Daley
@ 2018-11-28 6:43 ` Matthew Daley
2018-11-28 13:19 ` Julien Grall
0 siblings, 1 reply; 13+ messages in thread
From: Matthew Daley @ 2018-11-28 6:43 UTC (permalink / raw)
To: julien.grall; +Cc: Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
On Mon, 26 Nov 2018 at 14:07, Matthew Daley <mattd@bugfuzz.com> wrote:
>
> On Fri, 23 Nov 2018 at 09:51, Julien Grall <julien.grall@arm.com> wrote:
>>
>> Hi Matthew,
>>
>> Sorry for the late answer and thank you for testing the patch.
>>
>> On 11/13/18 10:43 PM, Matthew Daley wrote:
>> > On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
>> >> On 11/11/18 1:15 AM, Matthew Daley wrote:
>> > I gave this a go but unfortunately the same problem occurs (error
>> > -9s). Just to check nothing weird is happening I added a printk to
>> > check the value of __pa(init_secondary) in call_psci_cpu_on, giving
>> > 0xdfe00180.
>>
>> I have posted a patch that disables completely the relocation [1]. Could
>> you have a try and see whether it works now?
>>
>> Cheers,
>>
>> [1] https://lists.xen.org/archives/html/xen-devel/2018-11/msg02638.html
>>
>> --
>> Julien Grall
>
>
> Hi Julien,
>
> Unfortunately with this patch it appears to freeze after the call to switch_ttbr(ttbr).
>
> - Matthew
So looking at setup_pagetables, my understanding is that it sets up a
2MiB mapping for Xen's .text section (before then splitting it into
4KiB pages). However, GRUB is loading Xen in RAM at 0xB8736000 for me.
This isn't 2MiB aligned (and isn't in low memory either!), so I can't
see how this would work; it still requires relocation or a more
granular mapping, no?
- Matthew
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Problems building and running Xen on Hikey960
2018-11-28 6:43 ` Matthew Daley
@ 2018-11-28 13:19 ` Julien Grall
2018-11-28 15:46 ` Stefano Stabellini
0 siblings, 1 reply; 13+ messages in thread
From: Julien Grall @ 2018-11-28 13:19 UTC (permalink / raw)
To: Matthew Daley; +Cc: Konrad Rzeszutek Wilk, Stefano Stabellini, Xen-devel
Hi Matthew,
On 28/11/2018 06:43, Matthew Daley wrote:
> On Mon, 26 Nov 2018 at 14:07, Matthew Daley <mattd@bugfuzz.com> wrote:
>>
>> On Fri, 23 Nov 2018 at 09:51, Julien Grall <julien.grall@arm.com> wrote:
>>>
>>> Hi Matthew,
>>>
>>> Sorry for the late answer and thank you for testing the patch.
>>>
>>> On 11/13/18 10:43 PM, Matthew Daley wrote:
>>>> On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
>>>>> On 11/11/18 1:15 AM, Matthew Daley wrote:
>>>> I gave this a go but unfortunately the same problem occurs (error
>>>> -9s). Just to check nothing weird is happening I added a printk to
>>>> check the value of __pa(init_secondary) in call_psci_cpu_on, giving
>>>> 0xdfe00180.
>>>
>>> I have posted a patch that disables completely the relocation [1]. Could
>>> you have a try and see whether it works now?
>>>
>>> Cheers,
>>>
>>> [1] https://lists.xen.org/archives/html/xen-devel/2018-11/msg02638.html
>>>
>>> --
>>> Julien Grall
>>
>>
>> Hi Julien,
>>
>> Unfortunately with this patch it appears to freeze after the call to switch_ttbr(ttbr).
>>
>> - Matthew
>
> So looking at setup_pagetables, my understanding is that it sets up a
> 2MiB mapping for Xen's .text section (before then splitting it into
> 4KiB pages). However, GRUB is loading Xen in RAM at 0xB8736000 for me.
> This isn't 2MiB aligned (and isn't in low memory either!), so I can't
> see how this would work; it still requires relocation or a more
> granular mapping, no?
I missed the fact that we configure the PE/COFF header with 4KB alignment.
Therefore UEFI is free to load Xen at any address 4KB aligned.
I managed to reproduce it on some UEFI platform and came up with
the following patch:
commit 6b268dd4ae2b433ab7314347bada3889a483a8a5
Author: Julien Grall <julien.grall@arm.com>
Date: Wed Nov 28 12:59:27 2018 +0000
xen/arm64: Request UEFI to load Xen at a 2MB aligned address
At the moment, UEFI will load Xen at a 4KB aligned address while
bootloader using the Image format will load at 2MB aligned address.
Using 2MB aligned address will help to simplify Xen boot process. So
also switch UEFI to load at 2MB aligned address.
Signed-off-by: Julien Grall <julien.grall@arm.com>
---
This patch should allow us to remove a lot of code in the head.S to
cater the 4KB case.
diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
index 4589a37874..0d1b1202fa 100644
--- a/xen/arch/arm/arm64/head.S
+++ b/xen/arch/arm/arm64/head.S
@@ -175,7 +175,7 @@ optional_header:
extra_header_fields:
.quad 0 /* ImageBase */
- .long 0x1000 /* SectionAlignment (4 KByte) */
+ .long 0x200000 /* SectionAlignment (2 MByte) */
.long 0x8 /* FileAlignment */
.short 0 /* MajorOperatingSystemVersion */
.short 0 /* MinorOperatingSystemVersion */
So now arm64 always request to be loaded at 2MB aligned address. However it seems
that arm32 only request 4KB alignment. I will rework the patch to use 4KB mapping
when setting up Xen.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: Problems building and running Xen on Hikey960
2018-11-28 13:19 ` Julien Grall
@ 2018-11-28 15:46 ` Stefano Stabellini
2018-11-28 17:47 ` Julien Grall
0 siblings, 1 reply; 13+ messages in thread
From: Stefano Stabellini @ 2018-11-28 15:46 UTC (permalink / raw)
To: Julien Grall
Cc: Matthew Daley, Stefano Stabellini, Konrad Rzeszutek Wilk, Xen-devel
On Wed, 28 Nov 2018, Julien Grall wrote:
> Hi Matthew,
>
> On 28/11/2018 06:43, Matthew Daley wrote:
> > On Mon, 26 Nov 2018 at 14:07, Matthew Daley <mattd@bugfuzz.com> wrote:
> >>
> >> On Fri, 23 Nov 2018 at 09:51, Julien Grall <julien.grall@arm.com> wrote:
> >>>
> >>> Hi Matthew,
> >>>
> >>> Sorry for the late answer and thank you for testing the patch.
> >>>
> >>> On 11/13/18 10:43 PM, Matthew Daley wrote:
> >>>> On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
> >>>>> On 11/11/18 1:15 AM, Matthew Daley wrote:
> >>>> I gave this a go but unfortunately the same problem occurs (error
> >>>> -9s). Just to check nothing weird is happening I added a printk to
> >>>> check the value of __pa(init_secondary) in call_psci_cpu_on, giving
> >>>> 0xdfe00180.
> >>>
> >>> I have posted a patch that disables completely the relocation [1]. Could
> >>> you have a try and see whether it works now?
> >>>
> >>> Cheers,
> >>>
> >>> [1] https://lists.xen.org/archives/html/xen-devel/2018-11/msg02638.html
> >>>
> >>> --
> >>> Julien Grall
> >>
> >>
> >> Hi Julien,
> >>
> >> Unfortunately with this patch it appears to freeze after the call to switch_ttbr(ttbr).
> >>
> >> - Matthew
> >
> > So looking at setup_pagetables, my understanding is that it sets up a
> > 2MiB mapping for Xen's .text section (before then splitting it into
> > 4KiB pages). However, GRUB is loading Xen in RAM at 0xB8736000 for me.
> > This isn't 2MiB aligned (and isn't in low memory either!), so I can't
> > see how this would work; it still requires relocation or a more
> > granular mapping, no?
>
> I missed the fact that we configure the PE/COFF header with 4KB alignment.
> Therefore UEFI is free to load Xen at any address 4KB aligned.
>
> I managed to reproduce it on some UEFI platform and came up with
> the following patch:
well done!
> commit 6b268dd4ae2b433ab7314347bada3889a483a8a5
> Author: Julien Grall <julien.grall@arm.com>
> Date: Wed Nov 28 12:59:27 2018 +0000
>
> xen/arm64: Request UEFI to load Xen at a 2MB aligned address
>
> At the moment, UEFI will load Xen at a 4KB aligned address while
> bootloader using the Image format will load at 2MB aligned address.
>
> Using 2MB aligned address will help to simplify Xen boot process. So
> also switch UEFI to load at 2MB aligned address.
>
> Signed-off-by: Julien Grall <julien.grall@arm.com>
>
> ---
>
> This patch should allow us to remove a lot of code in the head.S to
> cater the 4KB case.
>
> diff --git a/xen/arch/arm/arm64/head.S b/xen/arch/arm/arm64/head.S
> index 4589a37874..0d1b1202fa 100644
> --- a/xen/arch/arm/arm64/head.S
> +++ b/xen/arch/arm/arm64/head.S
> @@ -175,7 +175,7 @@ optional_header:
>
> extra_header_fields:
> .quad 0 /* ImageBase */
> - .long 0x1000 /* SectionAlignment (4 KByte) */
> + .long 0x200000 /* SectionAlignment (2 MByte) */
> .long 0x8 /* FileAlignment */
> .short 0 /* MajorOperatingSystemVersion */
> .short 0 /* MinorOperatingSystemVersion */
>
>
> So now arm64 always request to be loaded at 2MB aligned address. However it seems
> that arm32 only request 4KB alignment. I will rework the patch to use 4KB mapping
> when setting up Xen.
>
> Cheers,
>
> --
> Julien Grall
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: Problems building and running Xen on Hikey960
2018-11-28 15:46 ` Stefano Stabellini
@ 2018-11-28 17:47 ` Julien Grall
0 siblings, 0 replies; 13+ messages in thread
From: Julien Grall @ 2018-11-28 17:47 UTC (permalink / raw)
To: Stefano Stabellini; +Cc: Matthew Daley, Konrad Rzeszutek Wilk, Xen-devel
Hi,
On 28/11/2018 15:46, Stefano Stabellini wrote:
> On Wed, 28 Nov 2018, Julien Grall wrote:
>> Hi Matthew,
>>
>> On 28/11/2018 06:43, Matthew Daley wrote:
>>> On Mon, 26 Nov 2018 at 14:07, Matthew Daley <mattd@bugfuzz.com> wrote:
>>>>
>>>> On Fri, 23 Nov 2018 at 09:51, Julien Grall <julien.grall@arm.com> wrote:
>>>>>
>>>>> Hi Matthew,
>>>>>
>>>>> Sorry for the late answer and thank you for testing the patch.
>>>>>
>>>>> On 11/13/18 10:43 PM, Matthew Daley wrote:
>>>>>> On Tue, 13 Nov 2018 at 02:01, Julien Grall <julien.grall@arm.com> wrote:
>>>>>>> On 11/11/18 1:15 AM, Matthew Daley wrote:
>>>>>> I gave this a go but unfortunately the same problem occurs (error
>>>>>> -9s). Just to check nothing weird is happening I added a printk to
>>>>>> check the value of __pa(init_secondary) in call_psci_cpu_on, giving
>>>>>> 0xdfe00180.
>>>>>
>>>>> I have posted a patch that disables completely the relocation [1]. Could
>>>>> you have a try and see whether it works now?
>>>>>
>>>>> Cheers,
>>>>>
>>>>> [1] https://lists.xen.org/archives/html/xen-devel/2018-11/msg02638.html
>>>>>
>>>>> --
>>>>> Julien Grall
>>>>
>>>>
>>>> Hi Julien,
>>>>
>>>> Unfortunately with this patch it appears to freeze after the call to switch_ttbr(ttbr).
>>>>
>>>> - Matthew
>>>
>>> So looking at setup_pagetables, my understanding is that it sets up a
>>> 2MiB mapping for Xen's .text section (before then splitting it into
>>> 4KiB pages). However, GRUB is loading Xen in RAM at 0xB8736000 for me.
>>> This isn't 2MiB aligned (and isn't in low memory either!), so I can't
>>> see how this would work; it still requires relocation or a more
>>> granular mapping, no?
>>
>> I missed the fact that we configure the PE/COFF header with 4KB alignment.
>> Therefore UEFI is free to load Xen at any address 4KB aligned.
>>
>> I managed to reproduce it on some UEFI platform and came up with
>> the following patch:
>
> well done!
Hmmm, it looks like some versions of GRUB are not happy with this patch. I am
not entirely sure why.
Anyway, I have a patch to use a smaller granule that should also fix another bug
at the same time. I send them tonight.
Cheers,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2018-11-28 17:47 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-11-08 7:07 Problems building and running Xen on Hikey960 Matthew Daley
2018-11-09 11:22 ` Julien Grall
2018-11-09 16:55 ` Omkar Bolla
2018-11-09 17:29 ` Julien Grall
2018-11-11 1:15 ` Matthew Daley
2018-11-12 13:01 ` Julien Grall
2018-11-13 22:43 ` Matthew Daley
2018-11-22 20:51 ` Julien Grall
2018-11-26 1:07 ` Matthew Daley
2018-11-28 6:43 ` Matthew Daley
2018-11-28 13:19 ` Julien Grall
2018-11-28 15:46 ` Stefano Stabellini
2018-11-28 17:47 ` Julien Grall
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.