Re: [PATCH 2/3] Documentation: riscv: Add documentation that describes the VM layout

From: Arnd Bergmann <arnd@arndb.de>
To: Alexandre Ghiti <alex@ghiti.fr>
Cc: David Hildenbrand <david@redhat.com>,
	Jonathan Corbet <corbet@lwn.net>,
	 Paul Walmsley <paul.walmsley@sifive.com>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	 Albert Ou <aou@eecs.berkeley.edu>,
	Andrey Ryabinin <aryabinin@virtuozzo.com>,
	 Alexander Potapenko <glider@google.com>,
	Dmitry Vyukov <dvyukov@google.com>,
	 "open list:DOCUMENTATION" <linux-doc@vger.kernel.org>,
	linux-riscv <linux-riscv@lists.infradead.org>,
	 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	kasan-dev <kasan-dev@googlegroups.com>,
	 linux-arch <linux-arch@vger.kernel.org>,
	Linux-MM <linux-mm@kvack.org>,
	 Linus Walleij <linus.walleij@linaro.org>
Subject: Re: [PATCH 2/3] Documentation: riscv: Add documentation that describes the VM layout
Date: Thu, 11 Mar 2021 09:42:01 +0100	[thread overview]
Message-ID: <CAK8P3a1+vSoEBqHPzj9S07B7h-Xuwvccpsh1pnn+1xJmS3UdbA@mail.gmail.com> (raw)
In-Reply-To: <236a9788-8093-9876-a024-b0ad0d672c72@ghiti.fr>

On Wed, Mar 10, 2021 at 8:12 PM Alex Ghiti <alex@ghiti.fr> wrote:
> Le 3/10/21 à 6:42 AM, Arnd Bergmann a écrit :
> > On Thu, Feb 25, 2021 at 12:56 PM Alex Ghiti <alex@ghiti.fr> wrote:
> >>
> >> Le 2/25/21 à 5:34 AM, David Hildenbrand a écrit :
> >>>                    |            |                  |         |> +
> >>> ffffffc000000000 | -256    GB | ffffffc7ffffffff |   32 GB | kasan
> >>>> +   ffffffcefee00000 | -196    GB | ffffffcefeffffff |    2 MB | fixmap
> >>>> +   ffffffceff000000 | -196    GB | ffffffceffffffff |   16 MB | PCI io
> >>>> +   ffffffcf00000000 | -196    GB | ffffffcfffffffff |    4 GB | vmemmap
> >>>> +   ffffffd000000000 | -192    GB | ffffffdfffffffff |   64 GB |
> >>>> vmalloc/ioremap space
> >>>> +   ffffffe000000000 | -128    GB | ffffffff7fffffff |  126 GB |
> >>>> direct mapping of all physical memory
> >>>
> >>> ^ So you could never ever have more than 126 GB, correct?
> >>>
> >>> I assume that's nothing new.
> >>>
> >>
> >> Before this patch, the limit was 128GB, so in my sense, there is nothing
> >> new. If ever we want to increase that limit, we'll just have to lower
> >> PAGE_OFFSET, there is still some unused virtual addresses after kasan
> >> for example.
> >
> > Linus Walleij is looking into changing the arm32 code to have the kernel
> > direct map inside of the vmalloc area, which would be another place
> > that you could use here. It would be nice to not have too many different
> > ways of doing this, but I'm not sure how hard it would be to rework your
> > code, or if there are any downsides of doing this.
>
> This was what my previous version did: https://lkml.org/lkml/2020/6/7/28.
>
> This approach was not welcomed very well and it fixed only the problem
> of the implementation of relocatable kernel. The second issue I'm trying
> to resolve here is to support both 3 and 4 level page tables using the
> same kernel without being relocatable (which would introduce performance
> penalty). I can't do it when the kernel mapping is in the vmalloc region
> since vmalloc region relies on PAGE_OFFSET which is different on both 3
> and 4 level page table and that would then require the kernel to be
> relocatable.

Ok, I see.

I suppose it might work if you moved the direct-map to the lowest
address and the vmalloc area (incorporating the kernel mapping,
modules, pio, and fixmap at fixed addresses) to the very top of the
address space, but you probably already considered and rejected
that for other reasons.

         Arnd