[PATCH] 64K page size

[PATCH] 64K page size

[Peter Watkins]

[-- Attachment #1: Type: text/plain, Size: 508 bytes --]

Hello,

There are a number of changes required to support larger page sizes, but 
this one I thought worth sending up right away.

The code in pgtable-64.h assumes TASK_SIZE is always bigger than a first 
level PGDIR_SIZE. This is not the case for 64K pages, where task size is 
40 bits (1TB) and a pgd entry can map 42 bits. This leads to 
USER_PTRS_PER_PGD being zero for 64K pages.

If there is interest in other changes for 64K pages, I can send more.

Signed-off-by: Peter Watkins <treestem@gmail.com>


[-- Attachment #2: patch-userptrs-2.6.18-rc1.txt --]
[-- Type: text/plain, Size: 622 bytes --]

diff --git a/include/asm-mips/pgtable-64.h b/include/asm-mips/pgtable-64.h
index e3db932..9ce72bd 100644
--- a/include/asm-mips/pgtable-64.h
+++ b/include/asm-mips/pgtable-64.h
@@ -93,8 +93,12 @@ #define PTRS_PER_PGD	((PAGE_SIZE << PGD_
 #define PTRS_PER_PMD	((PAGE_SIZE << PMD_ORDER) / sizeof(pmd_t))
 #define PTRS_PER_PTE	((PAGE_SIZE << PTE_ORDER) / sizeof(pte_t))
 
+#if PGDIR_SIZE >= TASK_SIZE
+#define USER_PTRS_PER_PGD       (1)
+#else
 #define USER_PTRS_PER_PGD	(TASK_SIZE / PGDIR_SIZE)
-#define FIRST_USER_ADDRESS	0
+#endif
+#define FIRST_USER_ADDRESS	0UL
 
 #define VMALLOC_START		MAP_BASE
 #define VMALLOC_END	\

Re: [PATCH] 64K page size

[Thiemo Seufer]

Peter Watkins wrote:
> Hello,
> 
> There are a number of changes required to support larger page sizes, but 
> this one I thought worth sending up right away.
> 
> The code in pgtable-64.h assumes TASK_SIZE is always bigger than a first 
> level PGDIR_SIZE. This is not the case for 64K pages, where task size is 
> 40 bits (1TB) and a pgd entry can map 42 bits. This leads to 
> USER_PTRS_PER_PGD being zero for 64K pages.
> 
> If there is interest in other changes for 64K pages, I can send more.

Please do so. :-)


Thiemo

Re: [PATCH] 64K page size

[Jonathan Day]

I am extremely interested in big pages (64K, etc), and
the sooner the better. If there is anything not
considered OK for immediate inclusion in the Linux
MIPS git tree, I would love to have a copy anyway.
Large pages will be necessary for some high-priority
work I'm doing, although stability at this point seems
to be an optional extra. (Hence why the patches are
much more important than whether they're actually
finished yet.)

Jonathan

--- Thiemo Seufer <ths@networkno.de> wrote:

> Peter Watkins wrote:
> > Hello,
> > 
> > There are a number of changes required to support
> larger page sizes, but 
> > this one I thought worth sending up right away.
> > 
> > The code in pgtable-64.h assumes TASK_SIZE is
> always bigger than a first 
> > level PGDIR_SIZE. This is not the case for 64K
> pages, where task size is 
> > 40 bits (1TB) and a pgd entry can map 42 bits.
> This leads to 
> > USER_PTRS_PER_PGD being zero for 64K pages.
> > 
> > If there is interest in other changes for 64K
> pages, I can send more.
> 
> Please do so. :-)
> 
> 
> Thiemo
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [PATCH] 64K page size

[Thiemo Seufer]

Jonathan Day wrote:
> I am extremely interested in big pages (64K, etc), and
> the sooner the better. If there is anything not
> considered OK for immediate inclusion in the Linux
> MIPS git tree, I would love to have a copy anyway.
> Large pages will be necessary for some high-priority
> work I'm doing, although stability at this point seems
> to be an optional extra. (Hence why the patches are
> much more important than whether they're actually
> finished yet.)

Biggest drawback (besides stability concerns and some broken userspace
programs) is the insane amount of memory it can take. Every tiny file
would currently take 64k in the page cache. There's some work going on
to collate such partially used pages into single ones, but that may
take a while to become usable.


Thiemo

Re: [PATCH] 64K page size

[Ralf Baechle]

On Wed, Aug 23, 2006 at 09:23:24AM -0700, Jonathan Day wrote:

> I am extremely interested in big pages (64K, etc), and
> the sooner the better. If there is anything not
> considered OK for immediate inclusion in the Linux
> MIPS git tree, I would love to have a copy anyway.
> Large pages will be necessary for some high-priority
> work I'm doing, although stability at this point seems
> to be an optional extra. (Hence why the patches are
> much more important than whether they're actually
> finished yet.)

64K pages are not the universal solution to world hunger.  They're a
tradeoff and usually one that is considered apropriate for full blown
supercomputers.  On smaller systems the memory overhead is likely to be
prohibitive.  The memory overhead problem is being worked on but it's
likely to be quite some time before this is finished and integrated.

Do we want to get them to work?  Of course, Linux/MIPS supports some
extremly large systems.  But aside of those 64K pagesize is rarely useful.

  Ralf

Re: [PATCH] 64K page size

[Jonathan Day]

The short answer is that I'm actually working on a
MIPS-based supercomputer. :)

The longer answer is that the group I'm working with
is linking an extremely large number of SB1-based MIPS
nodes together to build a very large multicast-based
cluster. Back-of-the-envelope calculations suggest
that large pages - although inefficient on memory per
node - would be highly efficient for what we have in
mind.

(You are correct that this will not involve solving
world hunger - although I'm always happy to be proven
wrong on such matters.)

Large pages are not the only technical issue that has
been bugging me - Linux clustering technology in
general is a long way from where I'd like it to be -
but I'm happy with the idea of someone solving one of
the potentially larger thorns that has been bugging me
for a while.

Jonathan Day


--- Ralf Baechle <ralf@linux-mips.org> wrote:
> 64K pages are not the universal solution to world
> hunger.  They're a
> tradeoff and usually one that is considered
> apropriate for full blown
> supercomputers.  On smaller systems the memory
> overhead is likely to be
> prohibitive.  The memory overhead problem is being
> worked on but it's
> likely to be quite some time before this is finished
> and integrated.
> 
> Do we want to get them to work?  Of course,
> Linux/MIPS supports some
> extremly large systems.  But aside of those 64K
> pagesize is rarely useful.
> 
>   Ralf
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

Re: [PATCH] 64K page size

[Ralf Baechle]

On Wed, Aug 23, 2006 at 11:15:49AM -0400, Peter Watkins wrote:

> There are a number of changes required to support larger page sizes, but 
> this one I thought worth sending up right away.
> 
> The code in pgtable-64.h assumes TASK_SIZE is always bigger than a first 
> level PGDIR_SIZE. This is not the case for 64K pages, where task size is 
> 40 bits (1TB) and a pgd entry can map 42 bits. This leads to 
> USER_PTRS_PER_PGD being zero for 64K pages.

Do you actually need large address space?  The kernel could allow 2, 3 and
4-level pagetables easily but doesn't give that choice currently.

  Ralf

Re: [PATCH] 64K page size

[Ralf Baechle]

On Wed, Aug 23, 2006 at 11:15:49AM -0400, Peter Watkins wrote:

> There are a number of changes required to support larger page sizes, but 
> this one I thought worth sending up right away.
> 
> The code in pgtable-64.h assumes TASK_SIZE is always bigger than a first 
> level PGDIR_SIZE. This is not the case for 64K pages, where task size is 
> 40 bits (1TB) and a pgd entry can map 42 bits. This leads to 
> USER_PTRS_PER_PGD being zero for 64K pages.

Thanks, applied.

  Ralf

Re: [PATCH] 64K page size

[Ralf Baechle]

On Wed, Aug 23, 2006 at 09:23:24AM -0700, Jonathan Day wrote:

> I am extremely interested in big pages (64K, etc), and
> the sooner the better. If there is anything not
> considered OK for immediate inclusion in the Linux
> MIPS git tree, I would love to have a copy anyway.
> Large pages will be necessary for some high-priority
> work I'm doing, although stability at this point seems
> to be an optional extra. (Hence why the patches are
> much more important than whether they're actually
> finished yet.)

We're getting very close to a 2.6.18 release and 64kB pages are still
quite experimental, so I'm putting all the 64kB pagesize related fixes
into the queue branch.  16kB by now has a few users, so I give it
higher priority.

I've been promised hugetlbfs since quite a while but that somehow never
materialized.  Or maybe that's just because the runtime generated TLB
exception handlers killed the patches ;-)

  Ralf

Re: [PATCH] 64K page size

[Yoichi Yuasa]

Hi Ralf,

On Tue, 29 Aug 2006 15:07:00 +0100
Ralf Baechle <ralf@linux-mips.org> wrote:

> On Wed, Aug 23, 2006 at 09:23:24AM -0700, Jonathan Day wrote:
> 
> > I am extremely interested in big pages (64K, etc), and
> > the sooner the better. If there is anything not
> > considered OK for immediate inclusion in the Linux
> > MIPS git tree, I would love to have a copy anyway.
> > Large pages will be necessary for some high-priority
> > work I'm doing, although stability at this point seems
> > to be an optional extra. (Hence why the patches are
> > much more important than whether they're actually
> > finished yet.)
> 
> We're getting very close to a 2.6.18 release and 64kB pages are still
> quite experimental, so I'm putting all the 64kB pagesize related fixes
> into the queue branch.  16kB by now has a few users, so I give it
> higher priority.

Which is your queue branch?
I want to test 64k page size on vr41xx.

Yoichi

Re: [PATCH] 64K page size

[Peter Watkins]

Ralf Baechle wrote:

>>The code in pgtable-64.h assumes TASK_SIZE is always bigger than a first 
>>level PGDIR_SIZE. This is not the case for 64K pages, where task size is 
>>40 bits (1TB) and a pgd entry can map 42 bits. This leads to 
>>USER_PTRS_PER_PGD being zero for 64K pages.
> 
> Do you actually need large address space?  The kernel could allow 2, 3 and
> 4-level pagetables easily but doesn't give that choice currently.
> 

Yes, in some cases. I'd like to benchmark the performance difference 
between 2 and 3 level pagetables, so the choice is valuable.

Re: [PATCH] 64K page size

[Ralf Baechle]

On Thu, Aug 31, 2006 at 12:48:09PM +0900, Yoichi Yuasa wrote:

> > We're getting very close to a 2.6.18 release and 64kB pages are still
> > quite experimental, so I'm putting all the 64kB pagesize related fixes
> > into the queue branch.  16kB by now has a few users, so I give it
> > higher priority.
> 
> Which is your queue branch?
> I want to test 64k page size on vr41xx.

It's just named "queue" in the usual git repository on linux-mips.org.

  Ralf