Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU

From: Julien Grall <julien@xen.org>
To: Elliott Mitchell <ehem+xen@m5p.com>, Corey Minyard <minyard@acm.org>
Cc: Stefano Stabellini <sstabellini@kernel.org>,
	Roman Shaposhnik <roman@zededa.com>,
	jeff.kubascik@dornerworks.com,
	Julien Grall <julien.grall@arm.com>,
	xen-devel@lists.xenproject.org,
	Stefano Stabellini <stefano.stabellini@xilinx.com>
Subject: Re: Troubles running Xen on Raspberry Pi 4 with 5.6.1 DomU
Date: Sat, 2 May 2020 20:43:53 +0100	[thread overview]
Message-ID: <00800979-6202-b2c1-7e1f-d05184d187a6@xen.org> (raw)
In-Reply-To: <20200502184836.GA96674@mattapan.m5p.com>

Hi Elliott,

On 02/05/2020 19:48, Elliott Mitchell wrote:
> On Sat, May 02, 2020 at 12:35:29PM -0500, Corey Minyard wrote:
>> On Sat, May 02, 2020 at 12:46:14PM +0100, Julien Grall wrote:
>>>
>>> On 02/05/2020 03:16, Corey Minyard wrote:
>>>>
>>>> Nope.  If you say 4096M of RAM, your issue is almost certainly DMA, but
>>>> it's not (just) the Linux code.  On the Raspberry Pi 4, several devices
>>>> cannot DMA to above 1024M of RAM, but Xen does not handle this.  The
>>>> 1024M of RAM is a limitation you will have to live with until Xen has a
>>>> fix.
>>>
>>> IIUC, dom0 would need to have some memory below 1GB for this to work, am I
>>> correct?
>>
>> No.  If I am understanding this correctly, all the memory in dom0 below
>> 1GB would have to be physically below 1GB.
>>
>> The Linux patch set starts at:
>>
>> https://lore.kernel.org/linux-iommu/20191015174616.GO13874@arrakis.emea.arm.com/t/
>>
> 
> Actually, things get worse.  What if someone wants to run an X-server in
> DomU and have a DomU accessing the graphics hardware?  Really needs to be
> a case of allocating DMA-capable memory means talking to Xen.

I am confused, if you passthrough a device to your DomU then you most 
likely going to want to protect using an IOMMU. So are you talking with 
or without IOMMU?

Lets imagine that you want to do without an IOMMU. The easiest way would 
be to direct-map your domain (e.g host physical == guest physical) 
because you require no modification in your guest. Only the toolstack 
and Xen would require modification.

Stefano has been working on a solution in the dom0less case (see [1]). 
The approach is to let the user select the region of RAM to use for a 
given guest.

> As pointed out in that discussion different boards are going to have the
> DMA-borderline in different places.  There could be enough variation that
> it needs to be settable at run time.  Then some boards might have some
> DMA devices which can access all memory, and some which cannot (full-DMA
> versus limited-DMA?).
> 
> 

Cheers,

[1] 
https://lore.kernel.org/xen-devel/91b9d1d9-6e6f-c8b9-55ac-a3477b20a17b@xen.org/T/#t

-- 
Julien Grall