* Is anonymous memory part of the page cache on Linux?
@ 2021-03-12 14:43 Peter Weber
2021-03-12 15:15 ` Matthew Wilcox
0 siblings, 1 reply; 7+ messages in thread
From: Peter Weber @ 2021-03-12 14:43 UTC (permalink / raw)
To: linux-mm
Hello!
Hopefully I'm asking here in the right place and don't disturb.
Is anonymous memory - i.e. program heap and stack - part of the page
cache on Linux? The documentation[1] of the kernel does not state that.
But the Wikipedia entry about page cache contains a graphic[2] (look at
the top right) which gives me the impression that 'malloc()' allocates
dynamic memory within the page cache.
Is it possible that the page cache serves as general foundation for
memory management on Linux? Taking into account that `mmap()` also
allows for anonymous mappings this could fit together?
I've asked this originally on Stackoverflow[3].
Thank you
Peter
[1]
https://www.kernel.org/doc/html/latest/admin-guide/mm/concepts.html#anonymous-memory
[2]
https://en.wikipedia.org/wiki/Page_cache#/media/File:The_Linux_Storage_Stack_Diagram.svg
[3] https://stackoverflow.com/q/66587345/1054324
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Is anonymous memory part of the page cache on Linux?
2021-03-12 14:43 Is anonymous memory part of the page cache on Linux? Peter Weber
@ 2021-03-12 15:15 ` Matthew Wilcox
2021-03-12 15:41 ` Peter Weber
0 siblings, 1 reply; 7+ messages in thread
From: Matthew Wilcox @ 2021-03-12 15:15 UTC (permalink / raw)
To: Peter Weber; +Cc: linux-mm
On Fri, Mar 12, 2021 at 03:43:33PM +0100, Peter Weber wrote:
> Hello!
> Hopefully I'm asking here in the right place and don't disturb.
>
> Is anonymous memory - i.e. program heap and stack - part of the page cache
> on Linux? The documentation[1] of the kernel does not state that. But the
> Wikipedia entry about page cache contains a graphic[2] (look at the top
> right) which gives me the impression that 'malloc()' allocates dynamic
> memory within the page cache.
The wikipedia diagram is wrong. Anonymous memory is not handled by the
page cache. Anonymous pages enter the storage stack via swap; they are
found in the page tables, sent to the swap cache and then written to
swap devices or swap files. Filesystems may get involved at that point,
but not always.
There are other weird things in the wikipedia diagram, like Direct I/O
being seemingly detached from applications, and not appearing to pass
through the VFS.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Is anonymous memory part of the page cache on Linux?
2021-03-12 15:15 ` Matthew Wilcox
@ 2021-03-12 15:41 ` Peter Weber
2021-03-12 22:45 ` Matthew Wilcox
0 siblings, 1 reply; 7+ messages in thread
From: Peter Weber @ 2021-03-12 15:41 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm
Thank you Matthew!
Am 2021-03-12 16:15, schrieb Matthew Wilcox:
> The wikipedia diagram is wrong. Anonymous memory is not handled by the
> page cache.
Is it roughly right to say, that the virtual memory uses page tables to
handle anonymous memory?
> Anonymous pages enter the storage stack via swap; they are
> found in the page tables, sent to the swap cache and then written to
> swap devices or swap files. Filesystems may get involved at that
> point,
> but not always.
And the page cache doesn't handle anonymous memory - even when it is
swapped?
> There are other weird things in the wikipedia diagram, like Direct I/O
> being seemingly detached from applications, and not appearing to pass
> through the VFS.
I'm not an expert. Maybe I should add at least note about that on
Wikipedia?
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Is anonymous memory part of the page cache on Linux?
2021-03-12 15:41 ` Peter Weber
@ 2021-03-12 22:45 ` Matthew Wilcox
2021-03-17 13:42 ` David Hildenbrand
0 siblings, 1 reply; 7+ messages in thread
From: Matthew Wilcox @ 2021-03-12 22:45 UTC (permalink / raw)
To: Peter Weber; +Cc: linux-mm
On Fri, Mar 12, 2021 at 04:41:55PM +0100, Peter Weber wrote:
> Thank you Matthew!
>
>
> Am 2021-03-12 16:15, schrieb Matthew Wilcox:
> > The wikipedia diagram is wrong. Anonymous memory is not handled by the
> > page cache.
>
> Is it roughly right to say, that the virtual memory uses page tables to
> handle anonymous memory?
I tend to deal with page cache and not anonymous memory, so I'm not sure
exactly how someone who's an expert would phrase it. Anonymous pages are
"handled" in a number of different ways -- they can be found on LRU lists
and they can be found through the page tables. It's all a bit ad-hoc as
far as I can tell ;-)
> > Anonymous pages enter the storage stack via swap; they are
> > found in the page tables, sent to the swap cache and then written to
> > swap devices or swap files. Filesystems may get involved at that point,
> > but not always.
>
> And the page cache doesn't handle anonymous memory - even when it is
> swapped?
Right. There's a swap cache, but that's not the same thing as the page
cache.
> > There are other weird things in the wikipedia diagram, like Direct I/O
> > being seemingly detached from applications, and not appearing to pass
> > through the VFS.
>
> I'm not an expert. Maybe I should add at least note about that on Wikipedia?
Maybe!
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Is anonymous memory part of the page cache on Linux?
2021-03-12 22:45 ` Matthew Wilcox
@ 2021-03-17 13:42 ` David Hildenbrand
0 siblings, 0 replies; 7+ messages in thread
From: David Hildenbrand @ 2021-03-17 13:42 UTC (permalink / raw)
To: Matthew Wilcox, Peter Weber; +Cc: linux-mm
On 12.03.21 23:45, Matthew Wilcox wrote:
> On Fri, Mar 12, 2021 at 04:41:55PM +0100, Peter Weber wrote:
>> Thank you Matthew!
>>
>>
>> Am 2021-03-12 16:15, schrieb Matthew Wilcox:
>>> The wikipedia diagram is wrong. Anonymous memory is not handled by the
>>> page cache.
>>
>> Is it roughly right to say, that the virtual memory uses page tables to
>> handle anonymous memory?
>
You'll have to distinguish between private and shared anonymous memory.
"private anonymous memory" -- mmap(MAP_ANONYMOUS | MAP_PRIVATE) -- is
usually completely managed using the process page tables. The only way
to get such pages mapped into another process is via fork(), whereby
page table are copied and COW (Copy On Write) applies.
Once dereferenced from all page tables, there are no other references
anymore; memory is handed back to the buddy as free memory, from where
it can be reused for other purposes. Of course, there are case where
there might be other references being taken (i.e., GUP, direct I/O ...),
but let's ignore that for now.
The essence for "private anonymous memory" is, that there is no other
way to get access to that memory besides the page tables. If you
MADV_DONTNEED virtual memory regions to zap the page table entries and
later reaccess the same virtual memory locations, always fresh memory
will be populated.
"shared anonymous memory" -- mmap(MAP_ANONYMOUS | MAP_SHARED) -- is a
little different and involves the page cache (it's pretty much shmem
without a user-visible fd). The only way to get such pages mapped into
another process is similarly via fork() (AFAIU), whereby page table are
copied and COW does _not_ apply.
Once dereferenced from all page tables, there is still a reference to
the page from the page cache. Only once evicted from the page cache, the
memory can be freed up.
The essence for "shared anonymous memory" is, that there are ways to get
access to that memory besides the page tables. If you MADV_DONTNEED such
virtual memory locations to zap the page table entries and later
reaccess the same virtual memory location again, the previous page will
be re-instantiated from the page cache. To evict the pages from the page
cache you would need MADV_REMOVE.
--
Thanks,
David / dhildenb
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Is anonymous memory part of the page cache on Linux?
2021-03-14 23:06 Wxz76
@ 2021-03-15 0:07 ` Matthew Wilcox
0 siblings, 0 replies; 7+ messages in thread
From: Matthew Wilcox @ 2021-03-15 0:07 UTC (permalink / raw)
To: Wxz76; +Cc: linux-mm, peter.weber
On Sun, Mar 14, 2021 at 11:06:12PM +0000, Wxz76@protonmail.com wrote:
> Hi Matthew and Peter,
>
> I had a few questions to clarify my understanding of the page/swap cache.
These are good questions to ask. I see the confusion, and I'm not
entirely sure how to sort it out for you, but let me try.
> > There's a swap cache, but that's not the same thing as the page cache.
>
> My understanding of the swap cache comes from:
>
> 1) Understanding the Linux Kernel by Bovet and Cesati:
>
> “The swap cache is implemented by the page cache data structures and procedures ” and
>
> “Pages in the swap cache are stored as every other page in the page cache, with the following special treatment:
>
> • The mapping field of the page descriptor is set to NULL.
>
> • The PG_swapcache flag of the page descriptor is set.
>
> • The private field stores the swapped-out page identifier associated with the page”
They're not wrong, but may be misleading. The swap cache reuses
(repurposes?) many of the same data structures used by the page cache.
In particular the address_space. It's still considered to be separate
from the page cache.
> 2) Understanding the Linux Virtual Memory Manager by Mel Gorman:
>
> “The swap cache is purely conceptual because it is simply a specialization of the page cache. The first principal difference between pages in the swap cache rather than the page cache is that pages in the swap
>
> cache always use swapper space as their address space in page→mapping. The second difference is that pages are added to the swap cache with add to swap cache(), shown in Figure 11.3, instead of add to page cache().”
>
> I understand that those books are more than ten years old, but is what they write no longer the case?
That one, being a little more specific, is now a little more out of date.
> Is the swap cache mechanism not a specialization of the page cache, and, if not, how are they different?
>
> > Anonymous memory is not handled by the page cache.
>
> > Anonymous pages enter the storage stack via swap; they are
>
> > found in the page tables, sent to the swap cache and then written to
>
> > swap devices or swap files.
>
> This is for the case of swapping out anonymous memory, but what about anonymous memory that is allocated dynamically with malloc/mmap: where is this memory coming from?
When the kernel needs to allocate a page for stack or malloc (whether it
be implemented through MAP_PRIVATE or brk()), it gets it from its pool of
free pages. It sets up the process's page tables to refer to that page,
and it adds the page to the LRU list (so it can be swapped out if the
pool of free pages runs low).
> When mmap opens files, it maps a process address spaces to a region in the page cache for the file, does it not?
Yes. This is the vm_area_struct which records the mapping from the
process address spce to the region of the file.
> Is this behavior not the same for allocating anonymous memory (minus dealing with a file)?
It's also a vm_area_struct, but several of the fields in the
vm_area_struct are used differently by an anonymous mapping than they
are by a file-based mapping.
The particularly interesting case that you didn't ask about is what
happens for mmap(MAP_PRIVATE) of a file. In that case, we set up for
a file-based mapping, but on a write fault, we allocate a new page,
copy from the page cache into the new page, set up the process page
table to point to this new page and put this new page into the LRU list,
so it can be swapped out if needed.
> I appreciate the help in clarifying this for me.
You're welcome! I hope this is useful.
^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: Is anonymous memory part of the page cache on Linux?
@ 2021-03-14 23:06 Wxz76
2021-03-15 0:07 ` Matthew Wilcox
0 siblings, 1 reply; 7+ messages in thread
From: Wxz76 @ 2021-03-14 23:06 UTC (permalink / raw)
To: willy; +Cc: linux-mm, peter.weber
[-- Attachment #1: Type: text/plain, Size: 2050 bytes --]
Hi Matthew and Peter,
I had a few questions to clarify my understanding of the page/swap cache.
> There's a swap cache, but that's not the same thing as the page cache.
My understanding of the swap cache comes from:
1) Understanding the Linux Kernel by Bovet and Cesati:
“The swap cache is implemented by the page cache data structures and procedures ” and
“Pages in the swap cache are stored as every other page in the page cache, with the following special treatment:
• The mapping field of the page descriptor is set to NULL.
• The PG_swapcache flag of the page descriptor is set.
• The private field stores the swapped-out page identifier associated with the page”
2) Understanding the Linux Virtual Memory Manager by Mel Gorman:
“The swap cache is purely conceptual because it is simply a specialization of the page cache. The first principal difference between pages in the swap cache rather than the page cache is that pages in the swap
cache always use swapper space as their address space in page→mapping. The second difference is that pages are added to the swap cache with add to swap cache(), shown in Figure 11.3, instead of add to page cache().”
I understand that those books are more than ten years old, but is what they write no longer the case?
Is the swap cache mechanism not a specialization of the page cache, and, if not, how are they different?
> Anonymous memory is not handled by the page cache.
> Anonymous pages enter the storage stack via swap; they are
> found in the page tables, sent to the swap cache and then written to
> swap devices or swap files.
This is for the case of swapping out anonymous memory, but what about anonymous memory that is allocated dynamically with malloc/mmap: where is this memory coming from?
When mmap opens files, it maps a process address spaces to a region in the page cache for the file, does it not?
Is this behavior not the same for allocating anonymous memory (minus dealing with a file)?
I appreciate the help in clarifying this for me.
wxz
[-- Attachment #2: Type: text/html, Size: 4843 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-03-17 13:47 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-12 14:43 Is anonymous memory part of the page cache on Linux? Peter Weber
2021-03-12 15:15 ` Matthew Wilcox
2021-03-12 15:41 ` Peter Weber
2021-03-12 22:45 ` Matthew Wilcox
2021-03-17 13:42 ` David Hildenbrand
2021-03-14 23:06 Wxz76
2021-03-15 0:07 ` Matthew Wilcox
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).