From mboxrd@z Thu Jan 1 00:00:00 1970 From: Peng Fan Subject: Re: Dom0 kernel panic when porting xen to new arm soc Date: Tue, 23 Jun 2015 21:03:17 +0800 Message-ID: <55895915.1010601@gmail.com> References: <5582D109.2000409@gmail.com> <1434639280.28264.42.camel@citrix.com> <55841799.6000406@gmail.com> <55853E20.6030700@gmail.com> <558573D0.6010207@gmail.com> <55857CF7.4000300@gmail.com> <5587E175.90704@citrix.com> <5587EEE6.7030706@gmail.com> <5588157E.2030701@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <5588157E.2030701@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Julien Grall , Ian Campbell Cc: xen-devel@lists.xen.org, Stefano Stabellini List-Id: xen-devel@lists.xenproject.org On 6/22/2015 10:02 PM, Julien Grall wrote: > On 22/06/15 12:17, Peng Fan wrote: >> I add debug log in this piece of code: >> void __init sanity_check_meminfo(void) >> >> { >> >> phys_addr_t memblock_limit = 0; >> int i, j, highmem = 0; >> phys_addr_t vmalloc_limit = __pa(vmalloc_min - 1) + 1; >> >> >> printk("vmalloc_min virt %x phys %x\n", vmalloc_min - 1, >> __pa(vmalloc_min - 1)); >> printk("vmalloc_limit %x\n", vmalloc_limit); >> >> If use 512M for Dom0, I found vmalloc_limit is 0xf800000, vmalloc_min is >> 0xef800000, This comes to a question, why __pa(vmalloc_min - 1) + will >> make vmalloc_limit only 0xf800000 which is less than 128M. the pv stub >> does some runtime fixup for virt_to_phys here. >> Since vmalloc_limit is small, then all other memory bank in my platform >> is recoginied as highmem, then arm_lowmem_limit is 0, then kernel panic: >> " >> dma_contiguous_reserve_area(size 1800000, base 00000000, limit 00000000) >> CMA: failed to reserve 32 MiB >> Memory policy: Data cache writealloc >> Kernel panic - not syncing: ERROR: Failed to allocate 0x2000 bytes below >> 0x0. >> " >> 0xffffffff - 0xef8000000 is about 264M. So I choose 256M as the Dom0 >> memory size. I do not have clear idea about this, current I am trying to >> use xl to boot DomU, so just use 256M for Dom0 here. > > There use to be some issue with the way how the offset between physical > and virtual address was computed. > > Do you use LPAE or short page table? Can you provide the xen log when > DOM0 is using 512MB? I did not enable LPAE for DOM0 kernel, use shor page table. Following is the full log from uboot to kernel with DOM0 512M: U-Boot 2015.04-rc4-00145-gf12a16e-dirty (Jun 19 2015 - 15:26:52) CPU: Freescale i.MX7D rev1.0 at 792 MHz CPU: Thermal invalid data, fuse: 0x1b800 CPU: Temperature: Can't find sensor device Reset cause: POR Board: MX7D 12x12 LPDDR3 ARM2 I2C: ready DRAM: 2 GiB PMIC: PFUZE300 DEV_ID=0x30 REV_ID=0x10 MMC: FSL_SDHC: 0, FSL_SDHC: 1, FSL_SDHC: 2 Display: MCIMX28LCD (800x480) Video: 800x480x24 In: serial Out: serial Err: serial switch to partitions #0, OK mmc0 is current device Net: FEC0 Error: FEC0 address not set. Normal Boot Hit any key to stop autoboot: 0 => setenv xen_addr_r 0x80000000 => setenv bootargs "console=dtuart dtuart=/soc/aips-bus@30800000/serial@30860000 dom0_mem=256M" => fatload mmc 0:1 0x80000000 xen.image reading xen.image 754544 bytes read in 129 ms (5.6 MiB/s) => setenv bootargs "console=dtuart dtuart=/soc/aips-bus@30800000/serial@30860000 dom0_mem=512M" => run loadfdt reading imx7d-12x12-lpddr3-arm2.dtb 40966 bytes read in 26 ms (1.5 MiB/s) => fatload mmc 0:1 0x84000000 zImage reading zImage 6250016 bytes read in 286 ms (20.8 MiB/s) => setenv kernel_addr_r 0x84000000 => setenv fdt_high 0xffffffff => setenv fdt_addr 0x83000000 => fdt addr ${fdt_addr} 0x40000 => fdt resize => fdt chosen => fdt set /chosen \#address-cells <1> => fdt set /chosen \#size-cells <1> => fdt set /chosen/module@0 compatible "xen,linux-zimage" "xen,multiboot-module" => fdt set /chosen/module@0 reg <${kernel_addr_r} 0x${filesize}> => fdt set /chosen/module@0 bootargs "console=hvc0 ro root=/dev/mmcblk0p2 clk_ignore_unused uart_from_osc loglevel=8 earlyprintk" => bootz ${xen_addr_r} - ${fdt_addr} Kernel image @ 0x80000000 [ 0x000000 - 0x101700 ] ## Flattened Device Tree blob at 83000000 Booting using the fdt blob at 0x83000000 reserving fdt memory region: addr=83000000 size=b000 Using Device Tree in place at 83000000, end 8300dfff Starting kernel ... - UART enabled - - CPU 00000000 booting - - Xen starting in Hyp mode - - Zero BSS - - Setting up control registers - - Turning on paging - - Ready - (XEN) Checking for initrd in /chosen (XEN) RAM: 0000000080000000 - 00000000ffffffff (XEN) (XEN) MODULE[0]: 0000000083000000 - 000000008300b000 Device Tree (XEN) MODULE[1]: 0000000084000000 - 00000000845f5e20 Kernel console=hvc0 ro root=/dev/mmcblk0p2 clk_ignore_unused uart_from_osc loglevel=8 earlyprintk (XEN) RESVD[0]: 0000000083000000 - 000000008300b000 (XEN) (XEN) Command line: console=dtuart dtuart=/soc/aips-bus@30800000/serial@30860000 dom0_mem=512M (XEN) Placing Xen at 0x00000000ffe00000-0x0000000100000000 (XEN) Update BOOTMOD_XEN from 0000000080000000-0000000080101701 => 00000000ffe00000-00000000fff01701 (XEN) Xen heap: 00000000fa000000-00000000fe000000 (16384 pages) (XEN) Dom heap: 507904 pages (XEN) Domain heap initialised (XEN) Platform: i.MX 7Dual (XEN) Looking for dtuart at "/soc/aips-bus@30800000/serial@30860000", options "" (XEN) imx-uart.c:226: xx 30860000 (XEN) imx-uart.c:98: ===============UFCR USR1 a01 2050 Xen 4.6-unstable (XEN) Xen version 4.6-unstable (Freenix@(none)) (arm-poky-linux-gnueabi-gcc (GCC) 4.8.2) debug=y Fri Jun 19 18:30:50 CST 2015 (XEN) Latest ChangeSet: Mon Jun 15 18:25:34 2015 +0800 git:c01e139-dirty (XEN) Processor: 410fc075: "ARM Limited", variant: 0x0, part 0xc07, rev 0x5 (XEN) 32-bit Execution: (XEN) Processor Features: 00001131:00011011 (XEN) Instruction Sets: AArch32 A32 Thumb Thumb-2 ThumbEE Jazelle (XEN) Extensions: GenericTimer Security (XEN) Debug Features: 02010555 (XEN) Auxiliary Features: 00000000 (XEN) Memory Model Features: 10101105 40000000 01240000 02102211 (XEN) ISA Features: 02101110 13112111 21232041 11112131 10011142 00000000 (XEN) Set Secondary entry to 00000000ffe0004c (0020004c) (XEN) Generic Timer IRQ: phys=30 hyp=26 virt=27 Freq: 8000 KHz (XEN) GICv2 initialization: (XEN) gic_dist_addr=0000000031001000 (XEN) gic_cpu_addr=0000000031002000 (XEN) gic_hyp_addr=0000000031004000 (XEN) gic_vcpu_addr=0000000031006000 (XEN) gic_maintenance_irq=25 (XEN) GICv2: 160 lines, 2 cpus, secure (IID 0100143b). (XEN) Using scheduler: SMP Credit Scheduler (credit) (XEN) imx-uart.c:117: imx_uart_init_postirq (XEN) Allocated console ring of 16 KiB. (XEN) VFP implementer 0x41 architecture 2 part 0x30 variant 0x7 rev 0x5 (XEN) Brought up 1 CPUs (XEN) P2M: 40-bit IPA (XEN) P2M: 3 levels with order-1 root, VTCR 0x80003558 (XEN) I/O virtualisation disabled (XEN) *** LOADING DOMAIN 0 *** (XEN) domain_build.c:1383: idle loop (XEN) Loading kernel from boot module @ 0000000084000000 (XEN) Allocating 1:1 mappings totalling 512MB for dom0: (XEN) BANK[0] 0x000000a0000000-0x000000c0000000 (512MB) (XEN) Loading zImage from 0000000084000000 to 00000000a7a00000-00000000a7ff5e20 (XEN) Allocating PPI 16 for event channel interrupt (XEN) Loading dom0 DTB to 0x00000000a8000000-0x00000000a8009de2 (XEN) regs -> pc = 0xa7a00000 (XEN) it is 32 bit machine (XEN) Scrubbing Free RAM on 1 nodes using 1 CPUs (XEN) ................done. (XEN) Initial low memory virq threshold set at 0x4000 pages. (XEN) Std. Loglevel: All (XEN) Guest Loglevel: All (XEN) *** Serial input -> DOM0 (type 'CTRL-a' three times to switch input to Xen) (XEN) Freed 284kB init memory. Uncompressing Linux... done, booting the kernel. Booting Linux on physical CPU 0x0 Linux version 3.14.38-02383-g5ccf32b-dirty (Freenix@linux-jyl1) (gcc version 4.8.2 (GCC) ) #49 SMP PREEMPT Mon Jun 22 14:17:14 CST 2015 CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c53c7d CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache Machine model: Freescale i.MX7 LPDDR3 12x12 ARM2 Board bootconsole [earlycon0] enabled vmalloc_min virt ef7fffff phys f7fffff dma_contiguous_reserve_area(size 1800000, base 00000000, limit 00000000) CMA: failed to reserve 32 MiB Memory policy: Data cache writealloc Kernel panic - not syncing: ERROR: Failed to allocate 0x2000 bytes below 0x0. > > Regards, > Regards, Peng.