Hand over of the Xen shared info page

From: Anastasiia Lukianenko <anastasiia_lukianenko@epam.com>
To: "xen-devel@lists.xenproject.org" <xen-devel@lists.xenproject.org>
Cc: Andrii Chepurnyi <Andrii_Chepurnyi@epam.com>,
	Oleksandr Andrushchenko <Oleksandr_Andrushchenko@epam.com>
Subject: Hand over of the Xen shared info page
Date: Thu, 13 May 2021 08:03:48 +0000	[thread overview]
Message-ID: <64bc6ab6ec387acebb40c1b4786dfda1050f9d50.camel@epam.com> (raw)

Hi all,

The problem described below concerns cases when a shared info page
needs to be handed over from one entity in the system to another, for
example, there is a bootloader or any other code that may run before
the guest OS' kernel.
Normally, to map the shared info page guests allocate a memory page
from their RAM and map the shared info on top of it. Specifically we
use XENMAPSPACE_shared_info memory space in  XENMEM_add_to_physmap
hypercall.  As the info page exists throughout the guest existence this
doesn't hurt the guest, but when the page gets out of accounting, e.g.
after bootloader jumps to Linux and the page is not handed over to it,
the mapped page becomes a problem.
Consider the case with U-boot bootloader which already has Xen support.
The U-boot’s Xen guest implementation allocates a shared info page
between Xen and the guest domain and U-boot uses domain's RAM address
space to create and map the shared info page by using
XENMEM_add_to_physmap hypercall [1].

After U-boot transfers control to the operating system (Linux, Android,
etc), the shared info page is still mapped in domain’s address space,
e.g. its RAM. So, after we leave U-boot, this page becomes just an
ordinary memory page from Linux POV while it is still a shared info
page from Xen POV. This can lead to undefined behavior, errors etc as
Xen can write something to the shared info page and when Linux tries to
use it - data corruption may happen.
This happens because there is no unmap function in Xen API to remove an
existing shared info page mapping. We could use only hypercall
XENMEM_remove_from_physmap which eventually will create a hole in the
domain's RAM address space which may also lead to guest’s crash while
accessing that memory.

We noticed this problem and the workaround was implemented using the
special GUEST_MAGIC memory region [2].

Now we want to make a proper solution based on GUEST_MAGIC_BASE, which
does not belong to the guest’s RAM address space [3]. Using the example
of how offsets for the console and xenstore are implemented, we can add
a new shared_info offset and increase the number of magic pages [4] and
implement related functionality, so there is a similar API to query
that magic page location as it is done for console PFN and others.
This approach would allow the use of the XENMEM_remove_from_physmap
hypercall without creating gaps in the RAM address space for Xen guest
OS [5].

[1] - 
https://github.com/u-boot/u-boot/blob/master/drivers/xen/hypervisor.c#L141
[2] - 
https://github.com/xen-troops/u-boot/commit/f759b151116af204a5ab02ace82c09300cd6233a
[3] - 
https://github.com/xen-project/xen/blob/master/xen/include/public/arch-arm.h#L432
[4] - 
https://github.com/xen-project/xen/blob/25849c8b16f2a5b7fcd0a823e80a5f1b590291f9/tools/libs/guest/xg_dom_arm.c#L29
[5] - 
https://github.com/xen-troops/u-boot/blob/android-master/drivers/xen/hypervisor.c#L162

Regards,
Anastasiia Lukianenko