Re: Keystone Issue

[CodeWiz2280 <codewiz2280@gmail.com>]

On Thu, Jun 4, 2020 at 6:16 AM Julien Grall <julien@xen.org> wrote:
>
> Hi,
>
> On 04/06/2020 10:08, Bertrand Marquis wrote:
> > I would have thought that linux would have need some memory, even small in the 32bit space in order to boot.
>
> Yes it needs some, but then they are switching to use the high memory
> alias after the MMU has been switch on.
>
>  From my understanding, the only difference is the page-tables will
> point to the high memory alias address rather than the low memory one.
> Linux will still be located at the same place but now accessed from the
> high memory alias rather than the low one.
>
> Note that AFAICT the secondary CPUs will still be brought-up using the
> low memory alias.
>
> > I could understand that some memory in the low address space needs to be reserved by Linux as DMA area for peripherals not supporting 36-bit addresses, but the whole low memory sounds like a big restriction.
> Many platforms have devices only supporting 32-bit DMA, but none of them
> require such aliasing. So this doesn't look to be the issue here.
>
> TBH, this code is only used by Keystone and switching address space is
> expensive (you have to turn off the MMU, updates page-tables, flush the
> cache...). I find hard to believe a developper would have come up with
> this complexity if it were possible to always use the low memory address
> range. It is even harder to believe Linux community would have accepted it.
>
> >
> > Would it be possible to have a bit more information on the “problem with peripherals” here ?
>
> I am curious as well, so I looked in more depth :). Going through the
> Linux history, one of the commit message [1] suggests they are switching
> to a coherent address space. The datasheet [2] (page 75) also confirm
> that the low region is not IO coherent.
>
> So I think you would not be able to do DMA without flush the cache which
> can be pretty expensive. For a PoC, it might be possible to force Linux
> flushing the area before and after each DMA request. This should be
> possible by marking the devices as not coherent.
>
> Although, I am not entirely sure if there is any fallout.
>
> @Dave, do you think it is possible for you to have a try? I can provide
> the patch for Linux to disable DMA coherency if possible.
I attempted to do that, where I removed the "dma-coherent" flags from
the device tree.  There are likely other issues, but the most glaring
problem that I ran into is that the ethernet does not work.  Eth0
shows up in ifconfig but there is no activity on it after a small
handful of message exchanges, whereas booting without Xen it seems to
work fine even if left in 32-bit mode (with the dma-coherent
disabled).  I don't know what implications behind the scenes there are
trying to stay in the lower 0x8000_0000 alias range either though.  I
would rather run it as intended by switching to the upper
0x8_0000_0000 alias region.

>
> For a proper solution, I think we need to implement something similar to
> what I wrote earlier.
>
> Cheers,
>
> [1] 5eb3da7246a5b2dfac9f38a7be62b1a0295584c7
> [2] https://www.ti.com/lit/ds/symlink/tci6638k2k.pdf?ts=1591183242813
>
>
> --
> Julien Grall