RE: Proposal for Porting Xen to Armv8-R64 - DraftA

[Wei Chen <Wei.Chen@arm.com>]

Hi Stefano,

> -----Original Message-----
> From: Xen-devel <xen-devel-bounces@lists.xenproject.org> On Behalf Of
> Stefano Stabellini
> Sent: 2022年3月8日 6:58
> To: Wei Chen <Wei.Chen@arm.com>
> Cc: Julien Grall <julien@xen.org>; xen-devel@lists.xenproject.org; Stefano
> Stabellini <sstabellini@kernel.org>; Bertrand Marquis
> <Bertrand.Marquis@arm.com>; Penny Zheng <Penny.Zheng@arm.com>; Henry Wang
> <Henry.Wang@arm.com>; nd <nd@arm.com>
> Subject: RE: Proposal for Porting Xen to Armv8-R64 - DraftA
> 
> On Mon, 7 Mar 2022, Wei Chen wrote:
> > > > On 01/03/2022 07:51, Wei Chen wrote:
> > > > >>> ### 1.2. Xen Challenges with PMSA Virtualization
> > > > >>> Xen is PMSA unaware Type-1 Hypervisor, it will need
> modifications to
> > > > run
> > > > >>> with an MPU and host multiple guest OSes.
> > > > >>>
> > > > >>> - No MMU at EL2:
> > > > >>>       - No EL2 Stage 1 address translation
> > > > >>>           - Xen provides fixed ARM64 virtual memory layout as
> basis
> > > of
> > > > >> EL2
> > > > >>>             stage 1 address translation, which is not applicable
> on
> > > > MPU
> > > > >> system,
> > > > >>>             where there is no virtual addressing. As a result,
> any
> > > > >> operation
> > > > >>>             involving transition from PA to VA, like ioremap,
> needs
> > > > >> modification
> > > > >>>             on MPU system.
> > > > >>>       - Xen's run-time addresses are the same as the link time
> > > > addresses.
> > > > >>>           - Enable PIC (position-independent code) on a real-
> time
> > > > target
> > > > >>>             processor probably very rare.
> > > > >>
> > > > >> Aside the assembly boot code and UEFI stub, Xen already runs at
> the
> > > > same
> > > > >> address as it was linked.
> > > > >>
> > > > >
> > > > > But the difference is that, base on MMU, we can use the same link
> > > > address
> > > > > for all platforms. But on MPU system, we can't do it in the same
> way.
> > > >
> > > > I agree that we currently use the same link address for all the
> > > > platforms. But this is also a problem when using MMU because EL2 has
> a
> > > > single TTBR.
> > > >
> > > > At the moment we are switching page-tables with the MMU which is not
> > > > safe. Instead we need to turn out the MMU off, switch page-tables
> and
> > > > then turn on the MMU. This means we need to have an identity mapping
> of
> > > > Xen in the page-tables. Assuming Xen is not relocated, the identity
> > > > mapping may clash with Xen (or the rest of the virtual address map).
> > > >
> > >
> > > Is this the same reason we create a dummy reloc section for EFI loader?
> > >
> > > > My initial idea was to enable PIC and update the relocation at boot
> > > > time. But this is a bit cumbersome to do. So now I am looking to
> have a
> > > > semi-dynamic virtual layout and find some place to relocate part of
> Xen
> > > > to use for CPU bring-up.
> > > >
> > > > Anyway, my point is we possibly could look at PIC if that could
> allow
> > > > generic Xen image.
> > > >
> > >
> > > I understand your concern. IMO, PIC is possible to do this, but
> obviously,
> > > it's not a small amount of work. And I want to hear some suggestions
> from
> > > Stefano, because he also has some solutions in previous thread.
> > >
> >
> > Can you have a look at the PIC discussion between Julien and me?
> > I think we may need some inputs from your view.
> 
> If we have to have a build-time device tree anyway, we could
> automatically generate the link address, together with other required
> addresses. There would little benefit to do PIC if we have to have a
> build-time device tree in any case.
> 
> On the other hand, if we could get rid of the build-time device tree
> altogether, then yes doing PIC provides some benefits. It would allow us
> to have single Xen binary working on multiple Cortex-R boards. However,
> I don't think that would be important from a user perspective. People
> will not install Ubuntu on a Cortex-R and apt-get xen.  The target is
> embedded: users will know from the start the board they will target, so
> it would not be a problem for them to build Xen for a specific board.
> ImageBuilder (or something like it) will still be required to generate
> boot scripts and boot info. In other words, although it would be
> convenient to produce a generic binary, it is not a must-have feature
> and I would consider it low-priority compared to others.

I tend to agree with your opinion. We can get some benefit from PIC,
but the priority may be low. We have encountered a problem when we're
trying to use EFI loader to boot xen.efi on v8R. Due to lack of relocation
capability, the EFI loader could not launch xen.efi on V8R. But Xen EFI
boot capability is a requirement of Arm EBBR [1]. In order to support Xen
EFI boot on V8R, may be we still need a partially supported PIC. Only some
boot code support PIC to make EFI relocation happy. This boot code will
help Xen to check its loaded address and relocate Xen image to Xen's
run-time address if need.

How about we place PIC support to TODO list for further discussion,
I don't think we can include so many items in day1 : ) 

[1]https://arm-software.github.io/ebbr/index.html

Cheers,
Wei Chen