Re: [PATCH] MIPS: Make sure ebase address is in KSEG0

[Huacai Chen <chenhc@lemote.com>]

Hi, Paul,

On Tue, Nov 26, 2019 at 7:44 AM Paul Burton <paulburton@kernel.org> wrote:
>
> Hi Huacai,
>
> On Sun, Nov 24, 2019 at 12:00:51PM +0800, Huacai Chen wrote:
> > On Sat, Nov 23, 2019 at 2:11 PM Jiaxun Yang <jiaxun.yang@flygoat.com> wrote:
> > > 于 2019年11月23日 GMT+08:00 下午1:08:38, Jiaxun Yang <jiaxun.yang@flygoat.com> 写到:
> > > >于 2019年11月23日 GMT+08:00 上午2:47:31, Paul Burton <paulburton@kernel.org>
> > > >写到:
> > > >>On Thu, Nov 21, 2019 at 07:30:20PM +0800, Huacai Chen wrote:
> > > >>> Dynamically allocated ebase address above 0x20000000 can be
> > > >triggered
> > > >>> by some special physical memory layout, or just by a "big kernel +
> > > >>big
> > > >>> initrd + big swiotlb" configuration.
> > > >>>
> > > >>> For MIPS32, ebase address above 0x20000000 is unusable, for MIPS64
> > > >it
> > > >>> is usable but this case is warned. However, this warning is useless
> > > >>> because it is unfixable in a specific system configuration. So we
> > > >>just
> > > >>> use CKSEG0 as a fallback.
> > > >>
> > > >>I'd prefer that we don't assume there's memory at physical address
> > > >zero
> > > >>- that property doesn't hold for all systems.
> > > >>
> > > >>How about the change I suggested previously over here:
> > > >>
> > > >>https://lore.kernel.org/linux-mips/20191108191149.bbq3h4xp4famsh2n@lantea.localdomain/
> > > >>
> > > >>Would that work for you?
> >
> > Now this patch has nothing to do with WG bit,
>
> Well, kind of. The 512MB limit is implicitly tied to the lack of a
> functional WG bit.
Yes, if CPU has no WG, we cannot use an ebase address above 512M, but
a CPU with WG also has problems, please see the comments:

            * It shouldn't generally be in XKPhys on MIPS64 to avoid
            * hitting a poorly defined exception base for Cache Errors.

So I think the warning is in order to tell us that we should avoid
ebase address above 512M. And if we don't think this is a problem then
let's simply remove the warning.

>
> > and also has nothing to do with Loongson, it just a problem with ebase
> > address -- on any MIPSr2 platforms where ebase is dynamically
> > allocated.
>
> Right, and the patch I proposed ensures that the allocation happens in
> the low 512MB for systems without the WG bit, ie. in memory we can
> access via kseg0. It just doesn't assume the memory is at physical
> address 0 - it might be, but on some systems there might not even be
> memory at 0.
>
> In practice we allocate memory bottom-up, so I'm not sure how you're
> managing to see problems here anyway - allocations this early should
> easily be within the low 512MB. If you could explain step by step what
> actually goes wrong on your system that would be helpful.
OK. Traditionally, Loongson-3's memory layout is like this:

0-240M: low memory
240-256M: reserved for firmware
256M-4G: holes, including PCI mem, MMIO registers, etc.
4G-...: high memory

In practise, the kernel can hold as large as 60M memory (because of
sparsemem), then initrd can be also as large as 60M (a full functional
rootfs), swiotlb can be as large as 128M. Initrd and swiotlb is
allocated before ebase, so in this extreme case, ebase address is
above 512M.

Huacai

>
> > In the comments it is said that ebase address above 0x20000000 (which
> > should be in XKphys) has problems to handle cache error. However, if
> > we really treat it as a problem, we should avoid it (not just a
> > warning); and if we don't think it is a problem, then we can remove
> > the warning (because the warning is unfixable in a specific system
> > configuration).
>
> The warning can probably go away if my patch is applied, since the
> allocation is constrained to the right memory region anyway.
>
> Thanks,
>     Paul