Address space in MIPS Linux kernel

Address space in MIPS Linux kernel

[Patrick Doyle]

Hello,
I am attempting to track down a sporadic bug in a Wi-Fi driver used by
my (32 bit) MIPS 4.14.115 kernel.

With the help of a judicious printk, I happened to notice a call to
memmove (wrapped in a call to NdisMoveMemory()) that looked like this:

[   56.855917] rtmp_ee_bin_write16: NdisMoveMemory(c157cf37, 82d49cb0, 2)

I am curious about the two address spaces highlighted in that call.  I
seem to recall (from a book I read a decade or two ago) that MIPS
treats caching and memory management differently based on the upper
few bits of the 32 bit physical address.  But I have no idea how that
maps into Linux's virtual memory management.  Nor am I sure where or
whom to ask.  So, I thought I would start by asking here.

What does an address like 0xc157cf37 mean to the kernel?
What does an address like 0x82d49cb0 mean to the kernel?

I am a little surprised to see that the destination address is odd,
while the source address is even... I'll go investigate that as well.

Thanks for any pointers to any documentation anybody can give me.  Of
course, direct answers are also welcome :-)

--wpd

Re: Address space in MIPS Linux kernel

[Patrick Doyle]

On Fri, Mar 6, 2020 at 6:25 AM <post@pfrst.de> wrote:
> Hi,
>
> Bit 31 being non-zero, these must be kernel-mode addresses anyway.
> 0xcXXXXXXX is in ksseg, TLB-mapped (only used to access "high-memory"
> outside the 0.5 GB unmapped kerenel-space, if at all)
> 0x8XXXXXXX is in kseg0, unmapped, cached (the usual kernel-address)
Hi Peter,
Thank you for your reply.  I managed to find my MIPS book finally, and read:

kuseg: 0x00000000-0x7FFFFFFF - User space mapped addresses
kseg0: 0x80000000-0x9FFFFFFF - Cached, "untranslated" address
kseg1: 0xA0000000-0xBFFFFFFF - Uncached, "untranslated" address
kseg2: 0xC0000000-0xFFFFFFFF - Kernel space mapped addresses (cached?)

So, I guess I'm wondering how the kernel ends up with some addresses
in kseg0 and some in kseg2.  Is there any significance to the fact
that one address is in kseg2 while the other is in kseg0.  I may be
chasing down a rabbit hole here, but, at this point, I don't know
what's causing my kernel panic, and I noticed something I didn't
understand (kseg2 vs kseg0), so I figured I would ask folks on this
list and learn something.  Maybe this will ultimately point me to the
source of my problem.  Maybe I will just learn something new.  Either
way, I win.

Thanks again for the reply.

--wpd

Re: Address space in MIPS Linux kernel

[Maciej W. Rozycki]

On Fri, 6 Mar 2020, Patrick Doyle wrote:

> Thank you for your reply.  I managed to find my MIPS book finally, and read:
> 
> kuseg: 0x00000000-0x7FFFFFFF - User space mapped addresses
> kseg0: 0x80000000-0x9FFFFFFF - Cached, "untranslated" address
> kseg1: 0xA0000000-0xBFFFFFFF - Uncached, "untranslated" address
> kseg2: 0xC0000000-0xFFFFFFFF - Kernel space mapped addresses (cached?)

 KSEG2 addresses will have the caching attribute set to one requested with 
the allocation that has made them.  At the hardware level the attribute is 
held in the respective TLB entries.

> So, I guess I'm wondering how the kernel ends up with some addresses
> in kseg0 and some in kseg2.  Is there any significance to the fact
> that one address is in kseg2 while the other is in kseg0.  I may be
> chasing down a rabbit hole here, but, at this point, I don't know
> what's causing my kernel panic, and I noticed something I didn't
> understand (kseg2 vs kseg0), so I figured I would ask folks on this
> list and learn something.  Maybe this will ultimately point me to the
> source of my problem.  Maybe I will just learn something new.  Either
> way, I win.

 KSEG2 addresses are handed out with virtual allocations, typically 
`vmalloc' and friends (also used for mapping kernel modules), but also 
`ioremap' and friends if hardware constraints mandate it.  KSEG0 addresses 
are handed out with regular allocations made with `kmalloc' and friends, 
and also `ioremap' and friends where feasible.  Also addresses coming from 
the kernel executable and the stack will be in KSEG0.

 This is for 32-bit MIPS/Linux configurations; 64-bit ones have additional 
segments, which will be used accordingly.

 HTH,

  Maciej

Re: Address space in MIPS Linux kernel

[Patrick Doyle]

On Fri, Mar 20, 2020 at 11:39 AM Maciej W. Rozycki <macro@linux-mips.org> wrote:
>  KSEG2 addresses are handed out with virtual allocations, typically
> `vmalloc' and friends (also used for mapping kernel modules), but also
> `ioremap' and friends if hardware constraints mandate it.  KSEG0 addresses
> are handed out with regular allocations made with `kmalloc' and friends,
> and also `ioremap' and friends where feasible.  Also addresses coming from
> the kernel executable and the stack will be in KSEG0.
>
>  This is for 32-bit MIPS/Linux configurations; 64-bit ones have additional
> segments, which will be used accordingly.
>
>  HTH,
>
>   Maciej
Thanks.  I tracked down the call to vmalloc.  I'm still have no clue
why we are triggering a kernel panic when/where we do, but it clearly
does not have anything to do with calling memmove() from KSEG2 space
to KSEG0 space, and I am now looking elsewhere (like what resources
may get allocated/locked on the way to that call to memmove() that
don't get freed/unlocked on the way back up, or what resources were
not locked that should have been).
--wpd