xen-devel.lists.xenproject.org archive mirror
 help / color / mirror / Atom feed
* ARMv8: New board bring up hangs in kernel start?
@ 2016-03-23 17:24 Dirk Behme
  2016-03-23 18:41 ` Konrad Rzeszutek Wilk
  2016-03-29 18:53 ` Julien Grall
  0 siblings, 2 replies; 11+ messages in thread
From: Dirk Behme @ 2016-03-23 17:24 UTC (permalink / raw)
  To: xen-devel

Hi,

trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I 
get [1] and then its hanging there.

I'd guess that it hangs due to missing timer interrupt, maybe missing 
interrupts at all?

Any hints how to debug this? Or where to look?

It might be possible that the board's firmware (arm-trusted-firmware 
based) doesn't configure anything correctly. Firmware is running at 
EL3, Xen at EL2. The same kernel is running fine without Xen.

Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c 
timer_interrupt() & vtimer_interrupt() but these don't seem to be 
called at all (?)

Best regards

Dirk

[1]

- UART enabled -
- CPU 00000000 booting -
- Current EL 00000008 -
- Xen starting at EL2 -
- Zero BSS -
- Setting up control registers -
- Turning on paging -
- Ready -
(XEN) Checking for initrd in /chosen
(XEN) RAM: 0000000048000000 - 000000007fffffff
(XEN)
(XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
(XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
(XEN)
(XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
(XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
(XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 => 
000000007fe00000-000000007ff12e01
(XEN) Domain heap initialised
(XEN) Platform: ARMv8 Cortex A57 64-bit eval board
(XEN) Taking dtuart configuration from /chosen/stdout-path
(XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
  Xen 4.7-unstable
(XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc 
(Linaro GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon Mar 
21 09:15:03 CET 2016
(XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893
(XEN) Processor: 411fd073: "ARM Limited", variant: 0x1, part 0xd07, 
rev 0x3
(XEN) 64-bit Execution:
(XEN)   Processor Features: 0000000000002222 0000000000000000
(XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
(XEN)     Extensions: FloatingPoint AdvancedSIMD
(XEN)   Debug Features: 0000000010305106 0000000000000000
(XEN)   Auxiliary Features: 0000000000000000 0000000000000000
(XEN)   Memory Model Features: 0000000000001124 0000000000000000
(XEN)   ISA Features:  0000000000011120 0000000000000000
(XEN) 32-bit Execution:
(XEN)   Processor Features: 00000131:00011011
(XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
(XEN)     Extensions: GenericTimer Security
(XEN)   Debug Features: 03010066
(XEN)   Auxiliary Features: 00000000
(XEN)   Memory Model Features: 10201105 40000000 01260000 02102211
(XEN)  ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
(XEN) Using PSCI-1.0 for SMP bringup
(XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 16660 KHz
(XEN) GICv2 initialization:
(XEN)         gic_dist_addr=00000000f1010000
(XEN)         gic_cpu_addr=00000000f1020000
(XEN)         gic_hyp_addr=00000000f1040000
(XEN)         gic_vcpu_addr=00000000f1060000
(XEN)         gic_maintenance_irq=25
(XEN) GICv2: 512 lines, 8 cpus, secure (IID 0200043b).
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Allocated console ring of 16 KiB.
(XEN) Brought up 1 CPUs
(XEN) P2M: 44-bit IPA with 44-bit PA
(XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594
(XEN) I/O virtualisation disabled
(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 0000000048200000
(XEN) Allocating 1:1 mappings totalling 512MB for dom0:
(XEN) BANK[0] 0x00000050000000-0x00000070000000 (512MB)
(XEN) Grant table range: 0x0000007fe00000-0x0000007fe5c000
(XEN) Loading zImage from 0000000048200000 to 
0000000050080000-0000000050a80000
(XEN) Allocating PPI 16 for event channel interrupt
(XEN) Loading dom0 DTB to 0x0000000058000000-0x000000005800568a
(XEN) Scrubbing Free RAM on 1 nodes using 1 CPUs
(XEN) .......done.
(XEN) Initial low memory virq threshold set at 0x4000 pages.
(XEN) Std. Loglevel: All
(XEN) Guest Loglevel: All
(XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch 
input to Xen)
(XEN) Freed 288kB init memory.
Booting Linux on physical CPU 0x0
Linux version 4.4.0+ (dirk@build) (gcc version 4.9.3 20150311 
(prerelease) (Linaro GCC 4.9-2015.03) ) #1 SMP PREEMPT Mon Mar 21 
09:12:13 CET 2016
Boot CPU: AArch64 Processor [411fd073]
debug: ignoring loglevel setting.
efi: Getting EFI parameters from FDT:
efi: UEFI not found.
cma: Reserved 16 MiB at 0x000000006f000000
On node 0 totalpages: 131072
   DMA zone: 2048 pages used for memmap
   DMA zone: 0 pages reserved
   DMA zone: 131072 pages, LIFO batch:31
psci: probing for conduit method from DT.
psci: PSCIv0.2 detected in firmware.
psci: Using standard PSCI v0.2 function IDs
psci: Trusted OS migration not required
Xen 4.7 support found
PERCPU: Embedded 20 pages/cpu @ffffffc01efc9000 s42112 r8192 d31616 u81920
pcpu-alloc: s42112 r8192 d31616 u81920 alloc=20*4096
pcpu-alloc: [0] 0
Detected PIPT I-cache on CPU0
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 
129024
Kernel command line: console=hvc0 ignore_loglevel
PID hash table entries: 2048 (order: 2, 16384 bytes)
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
software IO TLB [mem 0x69c00000-0x6dc00000] (64MB) mapped at 
[ffffffc019c00000-ffffffc01dbfffff]
Memory: 414204K/524288K available (6000K kernel code, 582K rwdata, 
2780K rodata, 704K init, 228K bss, 93700K reserved, 16384K cma-reserved)
Virtual kernel memory layout:
     vmalloc : 0xffffff8000000000 - 0xffffffbdbfff0000   (   246 GB)
     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB 
maximum)
               0xffffffbdc1400000 - 0xffffffbdc1c00000   (     8 MB 
actual)
     fixed   : 0xffffffbffa7fd000 - 0xffffffbffac00000   (  4108 KB)
     PCI I/O : 0xffffffbffae00000 - 0xffffffbffbe00000   (    16 MB)
     modules : 0xffffffbffc000000 - 0xffffffc000000000   (    64 MB)
     memory  : 0xffffffc000000000 - 0xffffffc020000000   (   512 MB)
       .init : 0xffffffc000916000 - 0xffffffc0009c6000   (   704 KB)
       .text : 0xffffffc000080000 - 0xffffffc0009151f4   (  8789 KB)
       .data : 0xffffffc0009c6000 - 0xffffffc000a57800   (   582 KB)
SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
Preemptible hierarchical RCU implementation.
  Build-time adjustment of leaf fanout to 64.
  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
NR_IRQS:64 nr_irqs:64 0
Architected cp15 timer(s) running at 16.66MHz (virt).
clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 
0x3d7a162dd, max_idle_ns: 440795202225 ns
sched_clock: 56 bits at 16MHz, resolution 60ns, wraps every 
2199023255551ns
Console: colour dummy device 80x25
console [hvc0] enabled
Calibrating delay loop (skipped), value calculated using timer 
frequency.. 33.32 BogoMIPS (lpj=66640)
pid_max: default: 32768 minimum: 301
Security Framework initialized
Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
ASID allocator initialised with 65536 entries
xen:grant_table: Grant tables using version 1 layout
Grant table initialized
xen:events: Using FIFO-based ABI
Xen: initializing cpu0
<hang>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-03-23 17:24 ARMv8: New board bring up hangs in kernel start? Dirk Behme
@ 2016-03-23 18:41 ` Konrad Rzeszutek Wilk
  2016-03-31 16:45   ` Stefano Stabellini
  2016-03-29 18:53 ` Julien Grall
  1 sibling, 1 reply; 11+ messages in thread
From: Konrad Rzeszutek Wilk @ 2016-03-23 18:41 UTC (permalink / raw)
  To: Dirk Behme, julien.grall, stefano.stabellini; +Cc: xen-devel

On Wed, Mar 23, 2016 at 06:24:40PM +0100, Dirk Behme wrote:
> Hi,

Hey,

CC-ing the ARM MAINTAINERs.

> 
> trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I get
> [1] and then its hanging there.
> 
> I'd guess that it hangs due to missing timer interrupt, maybe missing
> interrupts at all?
> 
> Any hints how to debug this? Or where to look?
> 
> It might be possible that the board's firmware (arm-trusted-firmware based)
> doesn't configure anything correctly. Firmware is running at EL3, Xen at
> EL2. The same kernel is running fine without Xen.
> 
> Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c
> timer_interrupt() & vtimer_interrupt() but these don't seem to be called at
> all (?)
> 
> Best regards
> 
> Dirk
> 
> [1]
> 
> - UART enabled -
> - CPU 00000000 booting -
> - Current EL 00000008 -
> - Xen starting at EL2 -
> - Zero BSS -
> - Setting up control registers -
> - Turning on paging -
> - Ready -
> (XEN) Checking for initrd in /chosen
> (XEN) RAM: 0000000048000000 - 000000007fffffff
> (XEN)
> (XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
> (XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
> (XEN)
> (XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
> (XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
> (XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 =>
> 000000007fe00000-000000007ff12e01
> (XEN) Domain heap initialised
> (XEN) Platform: ARMv8 Cortex A57 64-bit eval board
> (XEN) Taking dtuart configuration from /chosen/stdout-path
> (XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
>  Xen 4.7-unstable
> (XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc (Linaro
> GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon Mar 21 09:15:03
> CET 2016
> (XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893
> (XEN) Processor: 411fd073: "ARM Limited", variant: 0x1, part 0xd07, rev 0x3
> (XEN) 64-bit Execution:
> (XEN)   Processor Features: 0000000000002222 0000000000000000
> (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
> (XEN)     Extensions: FloatingPoint AdvancedSIMD
> (XEN)   Debug Features: 0000000010305106 0000000000000000
> (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
> (XEN)   Memory Model Features: 0000000000001124 0000000000000000
> (XEN)   ISA Features:  0000000000011120 0000000000000000
> (XEN) 32-bit Execution:
> (XEN)   Processor Features: 00000131:00011011
> (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
> (XEN)     Extensions: GenericTimer Security
> (XEN)   Debug Features: 03010066
> (XEN)   Auxiliary Features: 00000000
> (XEN)   Memory Model Features: 10201105 40000000 01260000 02102211
> (XEN)  ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
> (XEN) Using PSCI-1.0 for SMP bringup
> (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 16660 KHz
> (XEN) GICv2 initialization:
> (XEN)         gic_dist_addr=00000000f1010000
> (XEN)         gic_cpu_addr=00000000f1020000
> (XEN)         gic_hyp_addr=00000000f1040000
> (XEN)         gic_vcpu_addr=00000000f1060000
> (XEN)         gic_maintenance_irq=25
> (XEN) GICv2: 512 lines, 8 cpus, secure (IID 0200043b).
> (XEN) Using scheduler: SMP Credit Scheduler (credit)
> (XEN) Allocated console ring of 16 KiB.
> (XEN) Brought up 1 CPUs
> (XEN) P2M: 44-bit IPA with 44-bit PA
> (XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594
> (XEN) I/O virtualisation disabled
> (XEN) *** LOADING DOMAIN 0 ***
> (XEN) Loading kernel from boot module @ 0000000048200000
> (XEN) Allocating 1:1 mappings totalling 512MB for dom0:
> (XEN) BANK[0] 0x00000050000000-0x00000070000000 (512MB)
> (XEN) Grant table range: 0x0000007fe00000-0x0000007fe5c000
> (XEN) Loading zImage from 0000000048200000 to
> 0000000050080000-0000000050a80000
> (XEN) Allocating PPI 16 for event channel interrupt
> (XEN) Loading dom0 DTB to 0x0000000058000000-0x000000005800568a
> (XEN) Scrubbing Free RAM on 1 nodes using 1 CPUs
> (XEN) .......done.
> (XEN) Initial low memory virq threshold set at 0x4000 pages.
> (XEN) Std. Loglevel: All
> (XEN) Guest Loglevel: All
> (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
> Xen)
> (XEN) Freed 288kB init memory.
> Booting Linux on physical CPU 0x0
> Linux version 4.4.0+ (dirk@build) (gcc version 4.9.3 20150311 (prerelease)
> (Linaro GCC 4.9-2015.03) ) #1 SMP PREEMPT Mon Mar 21 09:12:13 CET 2016
> Boot CPU: AArch64 Processor [411fd073]
> debug: ignoring loglevel setting.
> efi: Getting EFI parameters from FDT:
> efi: UEFI not found.
> cma: Reserved 16 MiB at 0x000000006f000000
> On node 0 totalpages: 131072
>   DMA zone: 2048 pages used for memmap
>   DMA zone: 0 pages reserved
>   DMA zone: 131072 pages, LIFO batch:31
> psci: probing for conduit method from DT.
> psci: PSCIv0.2 detected in firmware.
> psci: Using standard PSCI v0.2 function IDs
> psci: Trusted OS migration not required
> Xen 4.7 support found
> PERCPU: Embedded 20 pages/cpu @ffffffc01efc9000 s42112 r8192 d31616 u81920
> pcpu-alloc: s42112 r8192 d31616 u81920 alloc=20*4096
> pcpu-alloc: [0] 0
> Detected PIPT I-cache on CPU0
> Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 129024
> Kernel command line: console=hvc0 ignore_loglevel
> PID hash table entries: 2048 (order: 2, 16384 bytes)
> Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
> Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
> software IO TLB [mem 0x69c00000-0x6dc00000] (64MB) mapped at
> [ffffffc019c00000-ffffffc01dbfffff]
> Memory: 414204K/524288K available (6000K kernel code, 582K rwdata, 2780K
> rodata, 704K init, 228K bss, 93700K reserved, 16384K cma-reserved)
> Virtual kernel memory layout:
>     vmalloc : 0xffffff8000000000 - 0xffffffbdbfff0000   (   246 GB)
>     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB maximum)
>               0xffffffbdc1400000 - 0xffffffbdc1c00000   (     8 MB actual)
>     fixed   : 0xffffffbffa7fd000 - 0xffffffbffac00000   (  4108 KB)
>     PCI I/O : 0xffffffbffae00000 - 0xffffffbffbe00000   (    16 MB)
>     modules : 0xffffffbffc000000 - 0xffffffc000000000   (    64 MB)
>     memory  : 0xffffffc000000000 - 0xffffffc020000000   (   512 MB)
>       .init : 0xffffffc000916000 - 0xffffffc0009c6000   (   704 KB)
>       .text : 0xffffffc000080000 - 0xffffffc0009151f4   (  8789 KB)
>       .data : 0xffffffc0009c6000 - 0xffffffc000a57800   (   582 KB)
> SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
> Preemptible hierarchical RCU implementation.
>  Build-time adjustment of leaf fanout to 64.
>  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
> RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
> NR_IRQS:64 nr_irqs:64 0
> Architected cp15 timer(s) running at 16.66MHz (virt).
> clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles:
> 0x3d7a162dd, max_idle_ns: 440795202225 ns
> sched_clock: 56 bits at 16MHz, resolution 60ns, wraps every 2199023255551ns
> Console: colour dummy device 80x25
> console [hvc0] enabled
> Calibrating delay loop (skipped), value calculated using timer frequency..
> 33.32 BogoMIPS (lpj=66640)
> pid_max: default: 32768 minimum: 301
> Security Framework initialized
> Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
> Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
> ASID allocator initialised with 65536 entries
> xen:grant_table: Grant tables using version 1 layout
> Grant table initialized
> xen:events: Using FIFO-based ABI
> Xen: initializing cpu0
> <hang>
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-03-23 17:24 ARMv8: New board bring up hangs in kernel start? Dirk Behme
  2016-03-23 18:41 ` Konrad Rzeszutek Wilk
@ 2016-03-29 18:53 ` Julien Grall
  2016-03-31 17:41   ` Dirk Behme
  1 sibling, 1 reply; 11+ messages in thread
From: Julien Grall @ 2016-03-29 18:53 UTC (permalink / raw)
  To: Dirk Behme, xen-devel

On 23/03/16 17:24, Dirk Behme wrote:
> Hi,

Hello Dirk,

Sorry for the late answer.

> trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I
> get [1] and then its hanging there.

The logs look normal.

Do you know where the kernel get stuck? You can press CTRL-a 3 times to 
get access to the Xen console and then press:
* 0 => dump Dom0 registers
* d => dump registers

> I'd guess that it hangs due to missing timer interrupt, maybe missing
> interrupts at all?
>
> Any hints how to debug this? Or where to look?
>
> It might be possible that the board's firmware (arm-trusted-firmware
> based) doesn't configure anything correctly. Firmware is running at EL3,
> Xen at EL2. The same kernel is running fine without Xen.
>
> Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c
> timer_interrupt() & vtimer_interrupt() but these don't seem to be called
> at all (?)

They should be called if the timer is configured correctly.

> Best regards
>
> Dirk
>
> [1]

[...]

 > (XEN) Checking for initrd in /chosen
 > (XEN) RAM: 0000000048000000 - 000000007fffffff
 > (XEN)
 > (XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
 > (XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
 > (XEN)
 > (XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
 > (XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
 > (XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 =>
 > 000000007fe00000-000000007ff12e01
 > (XEN) Domain heap initialised
 > (XEN) Platform: ARMv8 Cortex A57 64-bit eval board
 > (XEN) Taking dtuart configuration from /chosen/stdout-path
 > (XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
 >   Xen 4.7-unstable
 > (XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc
 > (Linaro GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon Mar 21
 > 09:15:03 CET 2016
 > (XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893

I can't find this changeset in tree. Can you provide your baseline 
commit and the list of patches you applied on top?

Also have you tried a newer version of Xen?

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-03-23 18:41 ` Konrad Rzeszutek Wilk
@ 2016-03-31 16:45   ` Stefano Stabellini
  0 siblings, 0 replies; 11+ messages in thread
From: Stefano Stabellini @ 2016-03-31 16:45 UTC (permalink / raw)
  To: Konrad Rzeszutek Wilk; +Cc: julien.grall, sstabellini, Dirk Behme, xen-devel

On Wed, 23 Mar 2016, Konrad Rzeszutek Wilk wrote:
> On Wed, Mar 23, 2016 at 06:24:40PM +0100, Dirk Behme wrote:
> > Hi,
> 
> Hey,
> 
> CC-ing the ARM MAINTAINERs.
> 
> > 
> > trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I get
> > [1] and then its hanging there.
> > 
> > I'd guess that it hangs due to missing timer interrupt, maybe missing
> > interrupts at all?
> > 
> > Any hints how to debug this? Or where to look?
> > 
> > It might be possible that the board's firmware (arm-trusted-firmware based)
> > doesn't configure anything correctly. Firmware is running at EL3, Xen at
> > EL2. The same kernel is running fine without Xen.
> > 
> > Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c
> > timer_interrupt() & vtimer_interrupt() but these don't seem to be called at
> > all (?)

I would add a few printks in arch/arm/xen/enlighten.c:xen_guest_init,
arch/arm/xen/enlighten.c:xen_percpu_init and maybe
arch/arm64/kernel/psci.c in Linux.

Try to see exactly where is getting stuck.


> > Best regards
> > 
> > Dirk
> > 
> > [1]
> > 
> > - UART enabled -
> > - CPU 00000000 booting -
> > - Current EL 00000008 -
> > - Xen starting at EL2 -
> > - Zero BSS -
> > - Setting up control registers -
> > - Turning on paging -
> > - Ready -
> > (XEN) Checking for initrd in /chosen
> > (XEN) RAM: 0000000048000000 - 000000007fffffff
> > (XEN)
> > (XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
> > (XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
> > (XEN)
> > (XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
> > (XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
> > (XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 =>
> > 000000007fe00000-000000007ff12e01
> > (XEN) Domain heap initialised
> > (XEN) Platform: ARMv8 Cortex A57 64-bit eval board
> > (XEN) Taking dtuart configuration from /chosen/stdout-path
> > (XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
> >  Xen 4.7-unstable
> > (XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc (Linaro
> > GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon Mar 21 09:15:03
> > CET 2016
> > (XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893
> > (XEN) Processor: 411fd073: "ARM Limited", variant: 0x1, part 0xd07, rev 0x3
> > (XEN) 64-bit Execution:
> > (XEN)   Processor Features: 0000000000002222 0000000000000000
> > (XEN)     Exception Levels: EL3:64+32 EL2:64+32 EL1:64+32 EL0:64+32
> > (XEN)     Extensions: FloatingPoint AdvancedSIMD
> > (XEN)   Debug Features: 0000000010305106 0000000000000000
> > (XEN)   Auxiliary Features: 0000000000000000 0000000000000000
> > (XEN)   Memory Model Features: 0000000000001124 0000000000000000
> > (XEN)   ISA Features:  0000000000011120 0000000000000000
> > (XEN) 32-bit Execution:
> > (XEN)   Processor Features: 00000131:00011011
> > (XEN)     Instruction Sets: AArch32 A32 Thumb Thumb-2 Jazelle
> > (XEN)     Extensions: GenericTimer Security
> > (XEN)   Debug Features: 03010066
> > (XEN)   Auxiliary Features: 00000000
> > (XEN)   Memory Model Features: 10201105 40000000 01260000 02102211
> > (XEN)  ISA Features: 02101110 13112111 21232042 01112131 00011142 00011121
> > (XEN) Using PSCI-1.0 for SMP bringup
> > (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 16660 KHz
> > (XEN) GICv2 initialization:
> > (XEN)         gic_dist_addr=00000000f1010000
> > (XEN)         gic_cpu_addr=00000000f1020000
> > (XEN)         gic_hyp_addr=00000000f1040000
> > (XEN)         gic_vcpu_addr=00000000f1060000
> > (XEN)         gic_maintenance_irq=25
> > (XEN) GICv2: 512 lines, 8 cpus, secure (IID 0200043b).
> > (XEN) Using scheduler: SMP Credit Scheduler (credit)
> > (XEN) Allocated console ring of 16 KiB.
> > (XEN) Brought up 1 CPUs
> > (XEN) P2M: 44-bit IPA with 44-bit PA
> > (XEN) P2M: 4 levels with order-0 root, VTCR 0x80043594
> > (XEN) I/O virtualisation disabled
> > (XEN) *** LOADING DOMAIN 0 ***
> > (XEN) Loading kernel from boot module @ 0000000048200000
> > (XEN) Allocating 1:1 mappings totalling 512MB for dom0:
> > (XEN) BANK[0] 0x00000050000000-0x00000070000000 (512MB)
> > (XEN) Grant table range: 0x0000007fe00000-0x0000007fe5c000
> > (XEN) Loading zImage from 0000000048200000 to
> > 0000000050080000-0000000050a80000
> > (XEN) Allocating PPI 16 for event channel interrupt
> > (XEN) Loading dom0 DTB to 0x0000000058000000-0x000000005800568a
> > (XEN) Scrubbing Free RAM on 1 nodes using 1 CPUs
> > (XEN) .......done.
> > (XEN) Initial low memory virq threshold set at 0x4000 pages.
> > (XEN) Std. Loglevel: All
> > (XEN) Guest Loglevel: All
> > (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to
> > Xen)
> > (XEN) Freed 288kB init memory.
> > Booting Linux on physical CPU 0x0
> > Linux version 4.4.0+ (dirk@build) (gcc version 4.9.3 20150311 (prerelease)
> > (Linaro GCC 4.9-2015.03) ) #1 SMP PREEMPT Mon Mar 21 09:12:13 CET 2016
> > Boot CPU: AArch64 Processor [411fd073]
> > debug: ignoring loglevel setting.
> > efi: Getting EFI parameters from FDT:
> > efi: UEFI not found.
> > cma: Reserved 16 MiB at 0x000000006f000000
> > On node 0 totalpages: 131072
> >   DMA zone: 2048 pages used for memmap
> >   DMA zone: 0 pages reserved
> >   DMA zone: 131072 pages, LIFO batch:31
> > psci: probing for conduit method from DT.
> > psci: PSCIv0.2 detected in firmware.
> > psci: Using standard PSCI v0.2 function IDs
> > psci: Trusted OS migration not required
> > Xen 4.7 support found
> > PERCPU: Embedded 20 pages/cpu @ffffffc01efc9000 s42112 r8192 d31616 u81920
> > pcpu-alloc: s42112 r8192 d31616 u81920 alloc=20*4096
> > pcpu-alloc: [0] 0
> > Detected PIPT I-cache on CPU0
> > Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 129024
> > Kernel command line: console=hvc0 ignore_loglevel
> > PID hash table entries: 2048 (order: 2, 16384 bytes)
> > Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
> > Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
> > software IO TLB [mem 0x69c00000-0x6dc00000] (64MB) mapped at
> > [ffffffc019c00000-ffffffc01dbfffff]
> > Memory: 414204K/524288K available (6000K kernel code, 582K rwdata, 2780K
> > rodata, 704K init, 228K bss, 93700K reserved, 16384K cma-reserved)
> > Virtual kernel memory layout:
> >     vmalloc : 0xffffff8000000000 - 0xffffffbdbfff0000   (   246 GB)
> >     vmemmap : 0xffffffbdc0000000 - 0xffffffbfc0000000   (     8 GB maximum)
> >               0xffffffbdc1400000 - 0xffffffbdc1c00000   (     8 MB actual)
> >     fixed   : 0xffffffbffa7fd000 - 0xffffffbffac00000   (  4108 KB)
> >     PCI I/O : 0xffffffbffae00000 - 0xffffffbffbe00000   (    16 MB)
> >     modules : 0xffffffbffc000000 - 0xffffffc000000000   (    64 MB)
> >     memory  : 0xffffffc000000000 - 0xffffffc020000000   (   512 MB)
> >       .init : 0xffffffc000916000 - 0xffffffc0009c6000   (   704 KB)
> >       .text : 0xffffffc000080000 - 0xffffffc0009151f4   (  8789 KB)
> >       .data : 0xffffffc0009c6000 - 0xffffffc000a57800   (   582 KB)
> > SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
> > Preemptible hierarchical RCU implementation.
> >  Build-time adjustment of leaf fanout to 64.
> >  RCU restricting CPUs from NR_CPUS=64 to nr_cpu_ids=1.
> > RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
> > NR_IRQS:64 nr_irqs:64 0
> > Architected cp15 timer(s) running at 16.66MHz (virt).
> > clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles:
> > 0x3d7a162dd, max_idle_ns: 440795202225 ns
> > sched_clock: 56 bits at 16MHz, resolution 60ns, wraps every 2199023255551ns
> > Console: colour dummy device 80x25
> > console [hvc0] enabled
> > Calibrating delay loop (skipped), value calculated using timer frequency..
> > 33.32 BogoMIPS (lpj=66640)
> > pid_max: default: 32768 minimum: 301
> > Security Framework initialized
> > Mount-cache hash table entries: 1024 (order: 1, 8192 bytes)
> > Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes)
> > ASID allocator initialised with 65536 entries
> > xen:grant_table: Grant tables using version 1 layout
> > Grant table initialized
> > xen:events: Using FIFO-based ABI
> > Xen: initializing cpu0
> > <hang>
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel
> 

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-03-29 18:53 ` Julien Grall
@ 2016-03-31 17:41   ` Dirk Behme
  2016-04-01 16:34     ` Julien Grall
  0 siblings, 1 reply; 11+ messages in thread
From: Dirk Behme @ 2016-03-31 17:41 UTC (permalink / raw)
  To: Julien Grall, xen-devel

Hello Julien,

On 29.03.2016 20:53, Julien Grall wrote:
> On 23/03/16 17:24, Dirk Behme wrote:
>> Hi,
>
> Hello Dirk,
>
> Sorry for the late answer.


Many thanks for answering at all :)


>> trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I
>> get [1] and then its hanging there.
>
> The logs look normal.
>
> Do you know where the kernel get stuck? You can press CTRL-a 3 times
> to get access to the Xen console and then press:
> * 0 => dump Dom0 registers
> * d => dump registers


Hmm, CTRL-a 3 times doesn't seem to work, either.

This does need working interrupts, too? I.e. that it doesn't work is 
an additional hint that anything with the interrupt handling might be 
wrong?

Maybe I should debug this, first.

It's handled by Xen's drivers/char/console.c / serial.c and the board 
specific UART device driver, correct?


>> I'd guess that it hangs due to missing timer interrupt, maybe missing
>> interrupts at all?
>>
>> Any hints how to debug this? Or where to look?
>>
>> It might be possible that the board's firmware (arm-trusted-firmware
>> based) doesn't configure anything correctly. Firmware is running at
>> EL3,
>> Xen at EL2. The same kernel is running fine without Xen.
>>
>> Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c
>> timer_interrupt() & vtimer_interrupt() but these don't seem to be
>> called
>> at all (?)
>
> They should be called if the timer is configured correctly.
>
>> Best regards
>>
>> Dirk
>>
>> [1]
>
> [...]
>
>  > (XEN) Checking for initrd in /chosen
>  > (XEN) RAM: 0000000048000000 - 000000007fffffff
>  > (XEN)
>  > (XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
>  > (XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
>  > (XEN)
>  > (XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
>  > (XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
>  > (XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 =>
>  > 000000007fe00000-000000007ff12e01
>  > (XEN) Domain heap initialised
>  > (XEN) Platform: ARMv8 Cortex A57 64-bit eval board
>  > (XEN) Taking dtuart configuration from /chosen/stdout-path
>  > (XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
>  >   Xen 4.7-unstable
>  > (XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc
>  > (Linaro GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon
> Mar 21
>  > 09:15:03 CET 2016
>  > (XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893
>
> I can't find this changeset in tree. Can you provide your baseline
> commit and the list of patches you applied on top?


This is

483ad4439f7fc7 xen-access: minor fixes

plus a local patch to support the board's serial port.


> Also have you tried a newer version of Xen?


I've switched to the recent master

a6f2cdb63 x86/hvm/viridian: keep APIC assist page mapped

now. No difference.

I'll have a deeper look into the interrupt configuration.

Is there anywhere some basic description which interrupts are supposed 
to be handled by XEN and which by the Linux kernel? I.e. how the ARM 
GIC should be configured regarding the distributor/CPU/virtual parts?

Best regards

Dirk


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-03-31 17:41   ` Dirk Behme
@ 2016-04-01 16:34     ` Julien Grall
  2016-04-04 15:44       ` Dirk Behme
  0 siblings, 1 reply; 11+ messages in thread
From: Julien Grall @ 2016-04-01 16:34 UTC (permalink / raw)
  To: Dirk Behme, xen-devel



On 31/03/16 18:41, Dirk Behme wrote:
> Hello Julien,

Hello Dirk,

> On 29.03.2016 20:53, Julien Grall wrote:
>> On 23/03/16 17:24, Dirk Behme wrote:
>>> trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I
>>> get [1] and then its hanging there.
>>
>> The logs look normal.
>>
>> Do you know where the kernel get stuck? You can press CTRL-a 3 times
>> to get access to the Xen console and then press:
>> * 0 => dump Dom0 registers
>> * d => dump registers
>
>
> Hmm, CTRL-a 3 times doesn't seem to work, either.
>
> This does need working interrupts, too? I.e. that it doesn't work is an
> additional hint that anything with the interrupt handling might be wrong?

The entry point for all the interrupts is do_IRQ. You can add a 
breakpoint there to know if you receive interrupts.

>
> Maybe I should debug this, first.
>
> It's handled by Xen's drivers/char/console.c / serial.c and the board
> specific UART device driver, correct?

The generic IRQ code (see do_IRQ) will dispatch the interrupt directly
to the interrupt handler you specific via setup_irq/request_irq.
Usually this handler is specific to the driver.

>>> I'd guess that it hangs due to missing timer interrupt, maybe missing
>>> interrupts at all?
>>>
>>> Any hints how to debug this? Or where to look?
>>>
>>> It might be possible that the board's firmware (arm-trusted-firmware
>>> based) doesn't configure anything correctly. Firmware is running at
>>> EL3,
>>> Xen at EL2. The same kernel is running fine without Xen.
>>>
>>> Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c
>>> timer_interrupt() & vtimer_interrupt() but these don't seem to be
>>> called
>>> at all (?)
>>
>> They should be called if the timer is configured correctly.
>>
>>> Best regards
>>>
>>> Dirk
>>>
>>> [1]
>>
>> [...]
>>
>>  > (XEN) Checking for initrd in /chosen
>>  > (XEN) RAM: 0000000048000000 - 000000007fffffff
>>  > (XEN)
>>  > (XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
>>  > (XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
>>  > (XEN)
>>  > (XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
>>  > (XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
>>  > (XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 =>
>>  > 000000007fe00000-000000007ff12e01
>>  > (XEN) Domain heap initialised
>>  > (XEN) Platform: ARMv8 Cortex A57 64-bit eval board
>>  > (XEN) Taking dtuart configuration from /chosen/stdout-path
>>  > (XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
>>  >   Xen 4.7-unstable
>>  > (XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc
>>  > (Linaro GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon
>> Mar 21
>>  > 09:15:03 CET 2016
>>  > (XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893
>>
>> I can't find this changeset in tree. Can you provide your baseline
>> commit and the list of patches you applied on top?
>
>
> This is
>
> 483ad4439f7fc7 xen-access: minor fixes
>
> plus a local patch to support the board's serial port.

Is it a patch to add earlyprintk or a completely new driver?

>
>
>> Also have you tried a newer version of Xen?
>
>
> I've switched to the recent master
>
> a6f2cdb63 x86/hvm/viridian: keep APIC assist page mapped
>
> now. No difference.
>
> I'll have a deeper look into the interrupt configuration.
>
> Is there anywhere some basic description which interrupts are supposed
> to be handled by XEN and which by the Linux kernel? I.e. how the ARM GIC
> should be configured regarding the distributor/CPU/virtual parts?

All the interrupts are taken by Xen. The function do_IRQ in Xen will 
dispatch the IRQ either to a guest or call a Xen specific handler.

Xen handles only a limited number of interrupt:
     * timers
     * UART
     * SMMU

The rest is either routed to guests or blacklisted by Xen.

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-04-01 16:34     ` Julien Grall
@ 2016-04-04 15:44       ` Dirk Behme
  2016-04-06 10:48         ` Julien Grall
  0 siblings, 1 reply; 11+ messages in thread
From: Dirk Behme @ 2016-04-04 15:44 UTC (permalink / raw)
  To: Julien Grall; +Cc: xen-devel

Hi Julien,

On 01.04.2016 18:34, Julien Grall wrote:
>
>
> On 31/03/16 18:41, Dirk Behme wrote:
>> Hello Julien,
>
> Hello Dirk,
>
>> On 29.03.2016 20:53, Julien Grall wrote:
>>> On 23/03/16 17:24, Dirk Behme wrote:
>>>> trying to bring up Xen on a new ARMv8 64-bit Cortex A57 eval board, I
>>>> get [1] and then its hanging there.
>>>
>>> The logs look normal.
>>>
>>> Do you know where the kernel get stuck? You can press CTRL-a 3 times
>>> to get access to the Xen console and then press:
>>> * 0 => dump Dom0 registers
>>> * d => dump registers
>>
>>
>> Hmm, CTRL-a 3 times doesn't seem to work, either.
>>
>> This does need working interrupts, too? I.e. that it doesn't work is an
>> additional hint that anything with the interrupt handling might be
>> wrong?
>
> The entry point for all the interrupts is do_IRQ. You can add a
> breakpoint there to know if you receive interrupts.
>
>>
>> Maybe I should debug this, first.
>>
>> It's handled by Xen's drivers/char/console.c / serial.c and the board
>> specific UART device driver, correct?
>
> The generic IRQ code (see do_IRQ) will dispatch the interrupt directly
> to the interrupt handler you specific via setup_irq/request_irq.
> Usually this handler is specific to the driver.
>
>>>> I'd guess that it hangs due to missing timer interrupt, maybe missing
>>>> interrupts at all?
>>>>
>>>> Any hints how to debug this? Or where to look?
>>>>
>>>> It might be possible that the board's firmware (arm-trusted-firmware
>>>> based) doesn't configure anything correctly. Firmware is running at
>>>> EL3,
>>>> Xen at EL2. The same kernel is running fine without Xen.
>>>>
>>>> Using a JTAG debugger I've put breakpoints into xen/arch/arm/time.c
>>>> timer_interrupt() & vtimer_interrupt() but these don't seem to be
>>>> called
>>>> at all (?)
>>>
>>> They should be called if the timer is configured correctly.
>>>
>>>> Best regards
>>>>
>>>> Dirk
>>>>
>>>> [1]
>>>
>>> [...]
>>>
>>>  > (XEN) Checking for initrd in /chosen
>>>  > (XEN) RAM: 0000000048000000 - 000000007fffffff
>>>  > (XEN)
>>>  > (XEN) MODULE[0]: 0000000048000000 - 00000000480058a2 Device Tree
>>>  > (XEN) MODULE[1]: 0000000048200000 - 0000000048c00000 Kernel
>>>  > (XEN)
>>>  > (XEN) Command line: console=dtuart dom0_mem=512M loglvl=all
>>>  > (XEN) Placing Xen at 0x000000007fe00000-0x0000000080000000
>>>  > (XEN) Update BOOTMOD_XEN from 0000000049000000-0000000049112e01 =>
>>>  > 000000007fe00000-000000007ff12e01
>>>  > (XEN) Domain heap initialised
>>>  > (XEN) Platform: ARMv8 Cortex A57 64-bit eval board
>>>  > (XEN) Taking dtuart configuration from /chosen/stdout-path
>>>  > (XEN) Looking for dtuart at "/soc/serial@e6e88000", options ""
>>>  >   Xen 4.7-unstable
>>>  > (XEN) Xen version 4.7-unstable (dirk@build) (aarch64-poky-linux-gcc
>>>  > (Linaro GCC 4.9-2015.03) 4.9.3 20150311 (prerelease)) debug=y Mon
>>> Mar 21
>>>  > 09:15:03 CET 2016
>>>  > (XEN) Latest ChangeSet: Tue Feb 9 09:37:15 2016 +0100 git:b0a2893
>>>
>>> I can't find this changeset in tree. Can you provide your baseline
>>> commit and the list of patches you applied on top?
>>
>>
>> This is
>>
>> 483ad4439f7fc7 xen-access: minor fixes
>>
>> plus a local patch to support the board's serial port.
>
> Is it a patch to add earlyprintk or a completely new driver?


Just earlyprintk.


>>> Also have you tried a newer version of Xen?
>>
>>
>> I've switched to the recent master
>>
>> a6f2cdb63 x86/hvm/viridian: keep APIC assist page mapped
>>
>> now. No difference.
>>
>> I'll have a deeper look into the interrupt configuration.
>>
>> Is there anywhere some basic description which interrupts are supposed
>> to be handled by XEN and which by the Linux kernel? I.e. how the ARM
>> GIC
>> should be configured regarding the distributor/CPU/virtual parts?
>
> All the interrupts are taken by Xen. The function do_IRQ in Xen will
> dispatch the IRQ either to a guest or call a Xen specific handler.
>
> Xen handles only a limited number of interrupt:
>      * timers
>      * UART
>      * SMMU
>
> The rest is either routed to guests or blacklisted by Xen.


Ok, thanks, that helps :) Once I have it working, maybe I post a patch 
to add this info to the documentation.


Just an other question:

On ARMv8 64-bit Xen is supposed to be started at EL2 *nonsecure*, correct?

It looks to me that the GICv2 on my board is already partly configured 
by the firmware at secure EL3. That does mean, whatever 
gicv2_dist_init() and gicv2_cpu_init() are supposed to do, they can't 
do it (completely) because they don't have access to the secure part 
of the GIC (?)


Best regards

Dirk




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-04-04 15:44       ` Dirk Behme
@ 2016-04-06 10:48         ` Julien Grall
  2016-04-16 17:39           ` Dirk Behme
  0 siblings, 1 reply; 11+ messages in thread
From: Julien Grall @ 2016-04-06 10:48 UTC (permalink / raw)
  To: Dirk Behme; +Cc: sstabellini, xen-devel



On 04/04/2016 16:44, Dirk Behme wrote:
> Hi Julien,

Hello Dirk,

> On 01.04.2016 18:34, Julien Grall wrote:
>>
>>
>> On 31/03/16 18:41, Dirk Behme wrote:
>>>> Also have you tried a newer version of Xen?
>>>
>>>
>>> I've switched to the recent master
>>>
>>> a6f2cdb63 x86/hvm/viridian: keep APIC assist page mapped
>>>
>>> now. No difference.
>>>
>>> I'll have a deeper look into the interrupt configuration.
>>>
>>> Is there anywhere some basic description which interrupts are supposed
>>> to be handled by XEN and which by the Linux kernel? I.e. how the ARM
>>> GIC
>>> should be configured regarding the distributor/CPU/virtual parts?
>>
>> All the interrupts are taken by Xen. The function do_IRQ in Xen will
>> dispatch the IRQ either to a guest or call a Xen specific handler.
>>
>> Xen handles only a limited number of interrupt:
>>      * timers
>>      * UART
>>      * SMMU
>>
>> The rest is either routed to guests or blacklisted by Xen.
>
>
> Ok, thanks, that helps :) Once I have it working, maybe I post a patch
> to add this info to the documentation.

That would be good. Thank you!

>
> Just an other question:
>
> On ARMv8 64-bit Xen is supposed to be started at EL2 *nonsecure*, correct?

That's right.

>
> It looks to me that the GICv2 on my board is already partly configured
> by the firmware at secure EL3. That does mean, whatever
> gicv2_dist_init() and gicv2_cpu_init() are supposed to do, they can't do
> it (completely) because they don't have access to the secure part of the
> GIC (?)

Which is normal, the secure part of the GIC should have already been 
initialized by the firmware running at secure EL3.

Do you know if Linux was able to initialize KVM on baremetal?

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-04-06 10:48         ` Julien Grall
@ 2016-04-16 17:39           ` Dirk Behme
  2016-04-18  8:17             ` Julien Grall
  0 siblings, 1 reply; 11+ messages in thread
From: Dirk Behme @ 2016-04-16 17:39 UTC (permalink / raw)
  To: Julien Grall; +Cc: sstabellini, xen-devel

Hi Julien,

On 06.04.2016 12:48, Julien Grall wrote:
>
> On 04/04/2016 16:44, Dirk Behme wrote:
>> Hi Julien,
>
> Hello Dirk,
>
>> On 01.04.2016 18:34, Julien Grall wrote:
>>>
>>>
>>> On 31/03/16 18:41, Dirk Behme wrote:
>>>>> Also have you tried a newer version of Xen?
>>>>
>>>>
>>>> I've switched to the recent master
>>>>
>>>> a6f2cdb63 x86/hvm/viridian: keep APIC assist page mapped
>>>>
>>>> now. No difference.
>>>>
>>>> I'll have a deeper look into the interrupt configuration.
>>>>
>>>> Is there anywhere some basic description which interrupts are
>>>> supposed
>>>> to be handled by XEN and which by the Linux kernel? I.e. how the ARM
>>>> GIC
>>>> should be configured regarding the distributor/CPU/virtual parts?
>>>
>>> All the interrupts are taken by Xen. The function do_IRQ in Xen will
>>> dispatch the IRQ either to a guest or call a Xen specific handler.
>>>
>>> Xen handles only a limited number of interrupt:
>>>      * timers
>>>      * UART
>>>      * SMMU
>>>
>>> The rest is either routed to guests or blacklisted by Xen.
>>
>>
>> Ok, thanks, that helps :) Once I have it working, maybe I post a patch
>> to add this info to the documentation.
>
> That would be good. Thank you!
>
>>
>> Just an other question:
>>
>> On ARMv8 64-bit Xen is supposed to be started at EL2 *nonsecure*,
>> correct?
>
> That's right.
>
>>
>> It looks to me that the GICv2 on my board is already partly configured
>> by the firmware at secure EL3. That does mean, whatever
>> gicv2_dist_init() and gicv2_cpu_init() are supposed to do, they
>> can't do
>> it (completely) because they don't have access to the secure part of
>> the
>> GIC (?)
>
> Which is normal, the secure part of the GIC should have already been
> initialized by the firmware running at secure EL3.


I'm using

https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/renesas/r8a7795.dtsi#n134

The special thing here is that it has the offsets 0x10000, 0x20000, 
0x40000 and 0x60000. Instead of the standard ones 0x1000, 0x2000, 
0x4000 and 0x6000.

Now, just by try & error, using [1] makes things working.

I'm not sure why this changes anything, though:

* To my understanding, the GIC register ranges are max 0x1000. So I'd 
think that the 0x2000 should be sufficient to map all necessary registers.

* I haven't observed any MMU fault. So it didn't look like any 
non-mapped register was accessed without this change.

Hmm ...

Best regards

Dirk

[1]

--- r8a7795.dtsi_orig	2016-04-16 19:33:09.698030462 +0200
+++ r8a7795.dtsi	2016-04-16 19:33:38.202029057 +0200
@@ -137,9 +137,9 @@
  			#address-cells = <0>;
  			interrupt-controller;
  			reg = <0x0 0xf1010000 0 0x1000>,
-			      <0x0 0xf1020000 0 0x2000>,
+			      <0x0 0xf1020000 0 0x20000>,
  			      <0x0 0xf1040000 0 0x20000>,
-			      <0x0 0xf1060000 0 0x2000>;
+			      <0x0 0xf1060000 0 0x20000>;
  			interrupts = <GIC_PPI 9
  					(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
  		};


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-04-16 17:39           ` Dirk Behme
@ 2016-04-18  8:17             ` Julien Grall
  2016-04-19 13:59               ` Dirk Behme
  0 siblings, 1 reply; 11+ messages in thread
From: Julien Grall @ 2016-04-18  8:17 UTC (permalink / raw)
  To: Dirk Behme; +Cc: Andre Przywara, sstabellini, Steve Capper, xen-devel



On 16/04/2016 18:39, Dirk Behme wrote:
> Hi Julien,

Hi Dirk,

>
> On 06.04.2016 12:48, Julien Grall wrote:
>>
>> On 04/04/2016 16:44, Dirk Behme wrote:
>>> Hi Julien,
> I'm using
>
> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/renesas/r8a7795.dtsi#n134
>
>
> The special thing here is that it has the offsets 0x10000, 0x20000,
> 0x40000 and 0x60000. Instead of the standard ones 0x1000, 0x2000, 0x4000
> and 0x6000.

Those offsets are normal, this platform seems to have a GIC using 
64KB-aligned region rather than the classic 4KB-aligned one.

> Now, just by try & error, using [1] makes things working.
>
> I'm not sure why this changes anything, though:
>
> * To my understanding, the GIC register ranges are max 0x1000. So I'd
> think that the 0x2000 should be sufficient to map all necessary registers.
>
> * I haven't observed any MMU fault. So it didn't look like any
> non-mapped register was accessed without this change.

Xen is checking if the platform is using an aliased GIC when the CPU 
interface size is 128KB (see gicv2_is_aliased and commit 
21550029f709072aacf3b90edd574e7d3021b400).

If you see "GICv2: Adjusting CPU interface base to..." then your GIC is 
aliased.

In any case, the device-tree does not correctly describe the hardware on 
your platform. Can you send a patch to fix the upstream device-tree?

Regards,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: ARMv8: New board bring up hangs in kernel start?
  2016-04-18  8:17             ` Julien Grall
@ 2016-04-19 13:59               ` Dirk Behme
  0 siblings, 0 replies; 11+ messages in thread
From: Dirk Behme @ 2016-04-19 13:59 UTC (permalink / raw)
  To: Julien Grall; +Cc: Andre Przywara, sstabellini, Steve Capper, xen-devel

On 18.04.2016 10:17, Julien Grall wrote:
>
>
> On 16/04/2016 18:39, Dirk Behme wrote:
>> Hi Julien,
>
> Hi Dirk,
>
>>
>> On 06.04.2016 12:48, Julien Grall wrote:
>>>
>>> On 04/04/2016 16:44, Dirk Behme wrote:
>>>> Hi Julien,
>> I'm using
>>
>> https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/arch/arm64/boot/dts/renesas/r8a7795.dtsi#n134
>>
>>
>>
>> The special thing here is that it has the offsets 0x10000, 0x20000,
>> 0x40000 and 0x60000. Instead of the standard ones 0x1000, 0x2000,
>> 0x4000
>> and 0x6000.
>
> Those offsets are normal, this platform seems to have a GIC using
> 64KB-aligned region rather than the classic 4KB-aligned one.
>
>> Now, just by try & error, using [1] makes things working.
>>
>> I'm not sure why this changes anything, though:
>>
>> * To my understanding, the GIC register ranges are max 0x1000. So I'd
>> think that the 0x2000 should be sufficient to map all necessary
>> registers.
>>
>> * I haven't observed any MMU fault. So it didn't look like any
>> non-mapped register was accessed without this change.
>
> Xen is checking if the platform is using an aliased GIC when the CPU
> interface size is 128KB (see gicv2_is_aliased and commit
> 21550029f709072aacf3b90edd574e7d3021b400).
>
> If you see "GICv2: Adjusting CPU interface base to..." then your GIC
> is aliased.
>
> In any case, the device-tree does not correctly describe the hardware
> on your platform. Can you send a patch to fix the upstream device-tree?


Done:

http://article.gmane.org/gmane.linux.kernel.renesas-soc/2951

Thanks!

Dirk



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2016-04-19 13:59 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-03-23 17:24 ARMv8: New board bring up hangs in kernel start? Dirk Behme
2016-03-23 18:41 ` Konrad Rzeszutek Wilk
2016-03-31 16:45   ` Stefano Stabellini
2016-03-29 18:53 ` Julien Grall
2016-03-31 17:41   ` Dirk Behme
2016-04-01 16:34     ` Julien Grall
2016-04-04 15:44       ` Dirk Behme
2016-04-06 10:48         ` Julien Grall
2016-04-16 17:39           ` Dirk Behme
2016-04-18  8:17             ` Julien Grall
2016-04-19 13:59               ` Dirk Behme

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).