Why does the kernel need a gig of VM?

Why does the kernel need a gig of VM?

[John Richard Moser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Can someone give me a layout of what exactly is up there?  I got the
basic idea

K 4G
A 3G
A 2G
A 1G

App has 3G, kernel has 1G at the top of VM on x86 (dunno about x86_64).

So what's the layout of that top 1G?  What's it all used for?  Is there
some obscene restriction of 1G of shared memory or something that gets
mapped up there?

How much does it need, and why?  What, if anything, is variable and
likely to do more than 10 or 15 megs of variation?

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+ps2hDd4aOud5P8RApCsAJ47J2Ye2YGljChTIETunRTUeM8kIQCfZyU2
Vm49zyAQONLuD4tScid3sSw=
=bxPu
-----END PGP SIGNATURE-----

Re: Why does the kernel need a gig of VM?

[Josh Boyer]

On Fri, 2005-01-28 at 15:06 -0500, John Richard Moser wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Can someone give me a layout of what exactly is up there?  I got the
> basic idea
> 
> K 4G
> A 3G
> A 2G
> A 1G
> 
> App has 3G, kernel has 1G at the top of VM on x86 (dunno about x86_64).
> 
> So what's the layout of that top 1G?  What's it all used for?  Is there
> some obscene restriction of 1G of shared memory or something that gets
> mapped up there?
> 
> How much does it need, and why?  What, if anything, is variable and
> likely to do more than 10 or 15 megs of variation?

Because of various reasons.  Normal kernel space virtual addresses
usually start at 0xc0000000, which is where the 3GiB userspace
restriction comes from.  

Then there is the vmalloc virtual address space, which usually starts at
a higher address than a normal kernel address.  Along the same lines are
ioremap addresses, etc.

Poke around in the header files.  I bet you'll find lots of reasons.

josh

Re: Why does the kernel need a gig of VM?

[John Richard Moser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Wow.

I'd heard that there was a way to set 3.5/0.5 GiB split, and that there
was a patch that removed the split and isolated the kernel (but that was
slow), so I was just curious about all this stuff with people screaming
about how tight 4G of VM is vs a half gig or a gig that can be freed up.

Josh Boyer wrote:
> On Fri, 2005-01-28 at 15:06 -0500, John Richard Moser wrote:
> 
>>-----BEGIN PGP SIGNED MESSAGE-----
>>Hash: SHA1
>>
>>Can someone give me a layout of what exactly is up there?  I got the
>>basic idea
>>
>>K 4G
>>A 3G
>>A 2G
>>A 1G
>>
>>App has 3G, kernel has 1G at the top of VM on x86 (dunno about x86_64).
>>
>>So what's the layout of that top 1G?  What's it all used for?  Is there
>>some obscene restriction of 1G of shared memory or something that gets
>>mapped up there?
>>
>>How much does it need, and why?  What, if anything, is variable and
>>likely to do more than 10 or 15 megs of variation?
> 
> 
> Because of various reasons.  Normal kernel space virtual addresses
> usually start at 0xc0000000, which is where the 3GiB userspace
> restriction comes from.  
> 
> Then there is the vmalloc virtual address space, which usually starts at
> a higher address than a normal kernel address.  Along the same lines are
> ioremap addresses, etc.
> 
> Poke around in the header files.  I bet you'll find lots of reasons.
> 
> josh
> 
> 

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFB+qUdhDd4aOud5P8RAmU8AJ9fRQi4A+yIVaXdv/oWlPIqObROPQCfUgvU
KAsRKxYgSTWVecLsZZCvXgE=
=v+fM
-----END PGP SIGNATURE-----

Re: Why does the kernel need a gig of VM?

[Bill Davidsen]

John Richard Moser wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Wow.
> 
> I'd heard that there was a way to set 3.5/0.5 GiB split, and that there
> was a patch that removed the split and isolated the kernel (but that was
> slow), so I was just curious about all this stuff with people screaming
> about how tight 4G of VM is vs a half gig or a gig that can be freed up.

The 4/4 split requires somewhat different logic than the others, but I 
believe that other splits could be specified at runtime instead of as a 
config option, should there ever be a need. From memory the 4/4 split 
needs table flushing on every kernel entry.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: Why does the kernel need a gig of VM?

[Oliver Neukum]

Am Freitag, 28. Januar 2005 21:42 schrieb Josh Boyer:
> Because of various reasons.  Normal kernel space virtual addresses
> usually start at 0xc0000000, which is where the 3GiB userspace
> restriction comes from.  
> 
> Then there is the vmalloc virtual address space, which usually starts at
> a higher address than a normal kernel address.  Along the same lines are
> ioremap addresses, etc.
> 
> Poke around in the header files.  I bet you'll find lots of reasons.

Probably, this some FAQ, but anyway. The kernel needs physical memory
present and accessible all the time from all contexts. This is mapped into
this area. All other RAM is called High Mem and needs to be specifically
mapped before it can be used from kernel space.

	Regards
		Oliver

Re: Why does the kernel need a gig of VM?

[Chris Friesen]

John Richard Moser wrote:

> So what's the layout of that top 1G?  What's it all used for?  Is there
> some obscene restriction of 1G of shared memory or something that gets
> mapped up there?
> 
> How much does it need, and why?  What, if anything, is variable and
> likely to do more than 10 or 15 megs of variation?

I'm guessing the high runners in the variable category are likely to be 
the page cache, all kinds of in-flight IO buffers, and such.

Chris

Re: Why does the kernel need a gig of VM?

[Andy Isaacson]

On Fri, Jan 28, 2005 at 03:06:15PM -0500, John Richard Moser wrote:
> Can someone give me a layout of what exactly is up there?  I got the
> basic idea
> 
> K 4G
> A 3G
> A 2G
> A 1G
> 
> App has 3G, kernel has 1G at the top of VM on x86 (dunno about x86_64).
> 
> So what's the layout of that top 1G?  What's it all used for?  Is there
> some obscene restriction of 1G of shared memory or something that gets
> mapped up there?

By default, the bottom 1G of physical memory is mapped into the 1G of
KVA.  (If you have less than 1G, it's all mapped.)  Thus, the TLB
remains valid across the user/kernel switch, which makes system calls
much faster.

The 4G/4G patches (google for the lwn.net overview) change this,
introducing a TLB flush on every syscall.  Better for some things
because you get more VA space, worse for most things because it's
slower.  (But it's "lots better for a few" versus "a little worse for
everybody", so the tradeoff is often worthwhile.) [1]

So the answer to your question is, "What's up there?  Memory.  All of it."
(Until you get to highmem.)

[1] The 4G/4G patch's *primary* goal is to increase the amount of KVA
    available to allow more "struct page" entries without exhausting
    lowmem.  Trying to manage 32GB or 64GB of physical memory with only
    896MB of lowmem is very difficult.  It has the additional advantage
    of allowing userland to mmap almost 4GB of stuff (as compared to
    almost 3GB without 4G/4G) which can be a nice win for database-type
    apps.

-andy