RE: Discussion of Xenheap problems on AArch64

[Henry Wang <Henry.Wang@arm.com>]

Hi Julien,

Sorry for the late reply, I kinda missed this email somehow....

Please see my inline reply ^^

> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> Julien Grall
> Sent: Monday, April 26, 2021 4:20 AM
> To: Henry Wang <Henry.Wang@arm.com>; sstabellini@kernel.org; xen-
> devel@lists.xenproject.org
> Cc: Wei Chen <Wei.Chen@arm.com>; Penny Zheng
> <Penny.Zheng@arm.com>; Bertrand Marquis <Bertrand.Marquis@arm.com>
> Subject: Re: Discussion of Xenheap problems on AArch64
> 
> 
> 
> On 21/04/2021 10:32, Henry Wang wrote:
> > Hi Julien,
> 
> Hi Henry,
> 
> >> -----Original Message-----
> >> From: Julien Grall <julien@xen.org>
> >> Sent: Wednesday, April 21, 2021 5:04 PM
> >> To: Henry Wang <Henry.Wang@arm.com>; sstabellini@kernel.org; xen-
> >> devel@lists.xenproject.org
> >> Cc: Wei Chen <Wei.Chen@arm.com>; Penny Zheng
> >> <Penny.Zheng@arm.com>; Bertrand Marquis
> <Bertrand.Marquis@arm.com>
> >> Subject: Re: Discussion of Xenheap problems on AArch64
> >>
> >>
> >>
> >> On 21/04/2021 07:28, Henry Wang wrote:
> >>> Hi,
> >>
> >> Hi Henry,
> >>
> >>>
> >>> We are trying to implement the static memory allocation on AArch64.
> Part
> >> of
> >>> this feature is the reserved heap memory allocation, where a specific
> range
> >> of
> >>> memory is reserved only for heap. In the development process, we
> found a
> >>> pitfall in current AArch64 setup_xenheap_mappings() function.
> >>>
> >>> According to a previous discussion in community
> >>> https://lore.kernel.org/xen-devel/20190216134456.10681-1-
> >> peng.fan@nxp.com/,
> >>> on AArch64, bootmem is initialized after setup_xenheap_mappings(),
> >>> setup_xenheap_mappings() may try to allocate memory before memory
> >> has been
> >>> handed over to the boot allocator. If the reserved heap memory
> allocation
> >> is
> >>> introduced, either of below 2 cases will trigger a crash:
> >>>
> >>> 1. If the reserved heap memory is at the end of the memory block list
> and
> >> the
> >>> gap between reserved and unreserved memory is bigger than 512GB,
> when
> >> we setup
> >>> mappings from the beginning of the memory block list, we will get OOM
> >> caused
> >>> by lack of pages in boot allocator. This is because the memory that is
> >> reserved
> >>> for heap has not been mapped and added to the boot allocator.
> >>>
> >>> 2. If we add the memory that is reserved for heap to boot allocator first,
> >> and
> >>> then setup mappings for banks in the memory block list, we may get a
> page
> >> which
> >>> has not been setup mapping, causing a data abort.
> >>
> >> There are a few issues with setup_xenheap_mappings(). I have been
> >> reworking the code on my spare time and started to upstream bits of it.
> >> A PoC can be found here:
> >>
> >> https://xenbits.xen.org/gitweb/?p=people/julieng/xen-
> >> unstable.git;a=shortlog;h=refs/heads/pt/dev
> >>
> >
> > Really great news! Thanks you very much for the information and your
> hard
> > work on the PoC :) I will start to go through your PoC code then.
> 
> I spent sometimes today to clean-up the PoC and sent a series on the ML
> (see [1]). This has been lightly tested so far.
> 
> Would you be able to give a try and let me know if it helps your problem?

Yes of course! I will start to test this series ^^ Thank you for your work!

> 
> For convenience, I have pushed a branch with the series applied here:
> 
> https://xenbits.xen.org/gitweb/?p=people/julieng/xen-
> unstable.git;a=shortlog;h=refs/heads/pt/rfc-v2
> 

Great, thanks!

> Cheers,
> 
> [1] https://lore.kernel.org/xen-devel/20210425201318.15447-1-
> julien@xen.org/
> 
> --
> Julien Grall