RE: [RFC PATCH 0/2] Introduce reserved Xenheap

[Henry Wang <Henry.Wang@arm.com>]

Hi Julien,

> -----Original Message-----
> From: Julien Grall <julien@xen.org>
> On 28/02/2022 07:12, Henry Wang wrote:
> > Hi Julien,
> 
> Hi Henry,
> 
> >> -----Original Message-----
> >> From: Julien Grall <julien@xen.org>
> >> Hi Henry,
> >>
> >> On 24/02/2022 01:30, Henry Wang wrote:
> >>> The reserved Xenheap, or statically configured Xenheap, refers to parts
> >>> of RAM reserved in the beginning for Xenheap. Like the static memory
> >>> allocation, such reserved Xenheap regions are reserved by configuration
> >>> in the device tree using physical address ranges.
> >>
> >> In Xen, we have the concept of domheap and xenheap. For Arm64 and
> x86
> >> they would be the same. But for Arm32, they would be different: xenheap
> >> is always mapped whereas domheap is separate.
> >>
> >> Skimming through the series, I think you want to use the region for both
> >> domheap and xenheap. Is that correct?
> >
> > Yes I think that would be correct, for Arm32, instead of using the full
> > `ram_pages` as the initial value of `heap_pages`, we want to use the
> > region specified in the device tree. But we are confused if this is the
> > correct (or preferred) way for Arm32, so in this series we only
> > implemented the reserved heap for Arm64.
> 
> That's an interesting point. When I skimmed through the series on
> Friday, my first thought was that for arm32 it would be only xenheap (so
> all the rest of memory is domheap).
> 
> However, Xen can allocate memory from domheap for its own purpose (e.g.
> we don't need contiguous memory, or for page-tables).
> 
> In a fully static environment, the domheap and xenheap are both going to
> be quite small. It would also be somewhat difficult for a user to size
> it. So I think, it would be easier to use the region you introduce for
> both domheap and xenheap.
> 
> Stefano, Bertrand, any opionions?
> 
> On a separate topic, I think we need some documentation explaining how a
> user can size the xenheap. How did you figure out for your setup?

Not sure if I fully understand the question. I will explain in two parts: I tested
this series on a dom0less (static mem) system on FVP_Base.
(1) For configuring the system, I followed the documentation I added in the
first patch in this series (docs/misc/arm/device-tree/booting.txt). The idea is
adding some static mem regions under the chosen node.

     chosen {
+        #xen,static-mem-address-cells = <0x2>;
+        #xen,static-mem-size-cells = <0x2>;
+        xen,static-mem = <0x8 0x80000000 0x0 0x00100000 0x8 0x90000000 0x0 0x08000000>;
     [...]
     }

(2) For verifying this series, what I did was basically playing with the region size
and number of the regions, adding printks and also see if the guests can boot
as expected when I change the xenheap size.

> 
> >>
> >> Furthemore, now that we are introducing more static region, it will get
> >> easier to overlap the regions by mistakes. I think we want to have some
> >> logic in Xen (or outside) to ensure that none of them overlaps. Do you
> >> have any plan for that?
> >
> > Totally agree with this idea, but before we actually implement the code,
> > we would like to firstly share our thoughts on this: One option could be to
> > add data structures to notes down these static memory regions when the
> > device tree is parsed, and then we can check if they are overlapped.
> 
> This should work.

Ack.

> 
> > Over
> > the long term (and this long term option is currently not in our plan),
> > maybe we can add something in the Xen toolstack for this usage?
> 
> When I read "Xen toolstack", I read the tools that will run in dom0. Is
> it what you meant?

Nonono, sorry for the misleading. I mean a build time tool that can run
on host (build machine) to generate/configure the Xen DTS for static
allocated memory. But maybe this tool can be placed in Xen tool or it can
be a separate tool that out of Xen's scope.

Anyway, this is just an idea as we find it is not easy for users to configure
so many static items manually.

> 
> >
> > Also, I am wondering if the overlapping check logic should be introduced
> > in this series. WDYT?
> 
> I would do that in a separate series.

Ack.

Kind regards,

Henry

> 
> Cheers,
> 
> --
> Julien Grall