Re: [PATCH v2 0/6] ARM: Add support for large kernel page (from 8K to 64K)

From: Arnd Bergmann <arnd@arndb.de>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Linux ARM <linux-arm-kernel@lists.infradead.org>,
	Gregory CLEMENT <gregory.clement@bootlin.com>,
	Russell King - ARM Linux admin <linux@armlinux.org.uk>,
	Thomas Petazzoni <thomas.petazzoni@bootlin.com>
Subject: Re: [PATCH v2 0/6] ARM: Add support for large kernel page (from 8K to 64K)
Date: Fri, 12 Jun 2020 14:49:06 +0200	[thread overview]
Message-ID: <CAK8P3a19pyVBwXS8pMgY3L29WyzhB8WmAJqXj5zyRh1aD8atMQ@mail.gmail.com> (raw)
In-Reply-To: <20200612122126.GC19739@gaia>

On Fri, Jun 12, 2020 at 2:21 PM Catalin Marinas <catalin.marinas@arm.com> wrote:
> On Fri, Jun 12, 2020 at 11:23:11AM +0200, Arnd Bergmann wrote:
> > On Thu, Jun 11, 2020 at 6:21 PM Russell King - ARM Linux admin
> > <linux@armlinux.org.uk> wrote:
> >
> > > If you increase the page size, then you need to increase the number
> > > of tables in a page, or suffer a huge amount of wasted memory taken
> > > for the page tables - going to an 8k page size means that the upper
> > > 4k of each page will not be used.  Going to 16k means the upper 12k
> > > won't be used.  And so on - as your software page size increases,
> > > the amount of memory wasted for each PTE table will increase
> > > unless you also increase the number of hardware 1st level entries
> > > pointing to each PTE page.  With 64k pages, 60k of each PTE page
> > > will remain unused.
> > >
> > > That isn't very efficient use of memory.
> >
> > I think this could be addressed by using the full page to contain
> > PTEs by making PTRS_PER_PTE larger and PTRS_PER_PGD
> > smaller, but there is an even bigger problem in the added memory
> > usage and I/O overhead for basically everything else: in any
> > sparsely populated memory mapped file or anonymous mapping,
> > the memory usage grows with the page size as well.
> >
> > I think Synology's vendor kernels for their NAS boxes have a
> > different hack to make large file systems work, by extending
> > the internal data types (I forgot which ones) to 64 bit. That is
> > probably more invasive to the generic kernel code, but should
> > be much more efficient and less invasive to ARM architecture
> > specific code.
>
> IIUC from Gregory's cover letter, the problem is page->index which is a
> pgoff_t, unsigned long. This limits us to a 32-bit page offsets, so a
> 44-bit actual file offset (16TB). It may be worth exploring this than
> hacking the page tables to pretend we have bigger page sizes.

Right, that's at least one type that needs to be changed, there
may be additional ones besides it. In Synology's patch, is also
a new rdx_t type that is defined the same way and used elsewhere,
at least for some SoCs (they use a maze of #ifdefs to merge
all the vendor kernels, and they also strip all code comments
and git history from the tarballs).

https://pastebin.com/e8C1zhzG has an attempt to split out
the relevant changes from the linux-3.10.105 tarball that they
use on Armada 385, see
https://sourceforge.net/projects/dsgpl/files/Synology%20NAS%20GPL%20Source/24922branch/armada38x-source/linux-3.10.x-bsp.txz/download
for the full kernel sources.

      Arnd

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel