Re: [Xen-devel] Errors with Loading Xen at a Certain Address

From: Julien Grall <julien.grall@arm.com>
To: Brian Woods <brian.woods@xilinx.com>
Cc: xen-devel <xen-devel@lists.xenproject.org>, nd <nd@arm.com>,
	Volodymyr Babchuk <Volodymyr_Babchuk@epam.com>,
	Stefano Stabellini <sstabellini@kernel.org>
Subject: Re: [Xen-devel] Errors with Loading Xen at a Certain Address
Date: Fri, 4 Oct 2019 10:49:28 +0100	[thread overview]
Message-ID: <edb70263-f98c-da0b-0e6a-859d6808b438@arm.com> (raw)
In-Reply-To: <20191004002502.GA9033@xilinx.com>

Hi Brian,

On 04/10/2019 01:25, Brian Woods wrote:
> On Thu, Oct 03, 2019 at 10:20:36PM +0100, Julien Grall wrote:
>> Hi Brian,
>>
>> On 10/3/19 9:24 PM, Brian Woods wrote:
>>> On Thu, Oct 03, 2019 at 07:23:23PM +0000, Julien Grall wrote:
>>> There's a WARN_ON() between the two debug printks calls I shared above.
>>
>> Looking at the log, the MFN seems to correspond to the one right after Xen
>> (0000000001400000 - 00000000015328f1) in memory.
>>
>> So it is normal to have the page given to the boot allocator. However, I am
>> not entirely sure which bit of init_done() is giving the page again to
>> xenheap.
>>
>> It is unlikely to be free_init_memory() because it deal with the init
>> section that is not at the end of the binary.
>>
>> This would leave discard_initial_modules() but there are a check to skip Xen
>> module.
>>
>> The call stack only print the address and not the symbol because it
>> unregistered the symbols for init. See unregister_init_virtual_memory().
>>
>> (XEN) Xen call trace:
>> (XEN)    [<000000000021c1a8>] page_alloc.c#free_heap_pages+0x1a8/0x614 (PC)
>> (XEN)    [<000000000021c1a8>] page_alloc.c#free_heap_pages+0x1a8/0x614 (LR)
>> (XEN)    [<000000000021e900>] page_alloc.c#init_heap_pages+0x3d4/0x564
>> (XEN)    [<000000000021eb24>] init_domheap_pages+0x94/0x9c
>> (XEN)    [<00000000002b83ec>] 00000000002b83ec
>> (XEN)    [<00000000002b8904>] 00000000002b8904
>> (XEN)    [<0000000000260a3c>] setup.c#init_done+0x10/0x20
>> (XEN)    [<00000000002b99ac>] 00000000002b99ac
>>
>> You should be able to use addr2line on the address with Xen binary.
>> I have the feeling this will point to discard_initial_modules() as this is
>> an init function and the symbol should not be printed.
>>
>> But, I can't see anything obviously wrong in the function... So I am not
>> entirely sure what could be the next steps.
>>
>> Cheers,
>>
>> -- 
>> Julien Grall
> 
> In the log, there's:
> (XEN) MODULE[0]: 0000000001400000 - 00000000015328f1 Xen
> (XEN) MODULE[1]: 00000000076d2000 - 00000000076dc080 Device Tree
> (XEN) MODULE[2]: 00000000076df000 - 0000000007fff364 Ramdisk
> (XEN) MODULE[3]: 0000000000080000 - 0000000003180000 Kernel
> (XEN)  RESVD[0]: 00000000076d2000 - 00000000076dc000
> (XEN)  RESVD[1]: 00000000076df000 - 0000000007fff364
> 
> Linux kernel	->   8_0000 - 318_0000
> Xen		-> 140_0000 - 153_28f1
> 
> There's something not quite right here... I'm guessing Xen was working
> at the address before because it was out of the "range" of the Linux
> kernel.  Now I guess I need to look into if it's a Xen or u-boot issue.

The loading address you wrote match the ones you seem to have requested in U-boot:

Filename 'yocto-Image'.
Load address: 0x80000

Filename 'xen-custom.ub'.
Load address: 0x1400000

But the size does not match the one you provided in the Device-Tree:

Bytes transferred = 18215424 (115f200 hex)

vs

0x0000000003180000 - 0x0000000000080000 = 0x3100000

This is always a risk when you write in advance the size of the binaries and 
location in the Device-Tree. If you are using tftp/load from FS, it is much less 
risky to provide a U-boot script that will generate the Xen DT node.

Cheers,

-- 
Julien Grall

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel