[PATCH v3 0/1] Documenting shmem as an exception case for the pagemap

[PATCH v3 0/1] Documenting shmem as an exception case for the pagemap

[Tiberiu A Georgescu]

This patch follows the discussions on previous documentation patch threads
[1][2]. It presents the exception case of shared memory management from the
pagemap's point of view. It briefly describes what is missing, why it is
missing and alternatives to the pagemap for page info retrieval in user
space.

In short, the kernel does not keep track of PTEs for swapped out shared
pages within the processes that references them. Thus, the proc/pid/pagemap
tool cannot print the swap destination of the shared memory pages, instead
setting the pagemap entry to zero for both non-allocated and swapped out
pages. This can create confusion for users who need information on swapped
out pages.

The reasons why maintaining the PTEs of all swapped out shared pages among
all processes while maintaining similar performance is not a trivial task,
or a desirable change, have been discussed extensively [1][3][4][5]. There
are also arguments for why this arguably missing information should
eventually be exposed to the user in either a future pagemap patch, or by
an alternative tool.

[1]: https://marc.info/?m=162878395426774
[2]: https://lore.kernel.org/lkml/20210920164931.175411-1-tiberiu.georgescu@nutanix.com/
[3]: https://lore.kernel.org/lkml/20210730160826.63785-1-tiberiu.georgescu@nutanix.com/
[4]: https://lore.kernel.org/lkml/20210807032521.7591-1-peterx@redhat.com/
[5]: https://lore.kernel.org/lkml/20210715201651.212134-1-peterx@redhat.com/

Tiberiu A Georgescu (1):
  Documentation: update pagemap with shmem exceptions

 Documentation/admin-guide/mm/pagemap.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

-- 
2.33.0.363.g4c719308ce

[PATCH v3 1/1] Documentation: update pagemap with shmem exceptions

[Tiberiu A Georgescu]

Mentioning the current missing information in the pagemap and alternatives
on how to retrieve it, in case someone stumbles upon unexpected behaviour.

Signed-off-by: Tiberiu A Georgescu <tiberiu.georgescu@nutanix.com>
Reviewed-by: Ivan Teterevkov <ivan.teterevkov@nutanix.com>
Reviewed-by: Florian Schmidt <florian.schmidt@nutanix.com>
Reviewed-by: Carl Waldspurger <carl.waldspurger@nutanix.com>
Reviewed-by: Jonathan Davies <jonathan.davies@nutanix.com>
---
 Documentation/admin-guide/mm/pagemap.rst | 22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)

diff --git a/Documentation/admin-guide/mm/pagemap.rst b/Documentation/admin-guide/mm/pagemap.rst
index fb578fbbb76c..4581527c07ae 100644
--- a/Documentation/admin-guide/mm/pagemap.rst
+++ b/Documentation/admin-guide/mm/pagemap.rst
@@ -196,6 +196,28 @@ you can go through every map in the process, find the PFNs, look those up
 in kpagecount, and tally up the number of pages that are only referenced
 once.
 
+Exceptions for Shared Memory
+============================
+
+Page table entries for shared pages are cleared when the pages are zapped or
+swapped out. This makes swapped out pages indistinguishable from never-allocated
+ones.
+
+In kernel space, the swap location can still be retrieved from the page cache.
+However, values stored only on the normal PTE get lost irretrievably when the
+page is swapped out (i.e. SOFT_DIRTY).
+
+In user space, whether the page is present, swapped or none can be deduced with
+the help of lseek and/or mincore system calls.
+
+lseek() can differentiate between accessed pages (present or swapped out) and
+holes (none/non-allocated) by specifying the SEEK_DATA flag on the file where
+the pages are backed. For anonymous shared pages, the file can be found in
+``/proc/pid/map_files/``.
+
+mincore() can differentiate between pages in memory (present, including swap
+cache) and out of memory (swapped out or none/non-allocated).
+
 Other notes
 ===========
 
-- 
2.33.0.363.g4c719308ce

Re: [PATCH v3 1/1] Documentation: update pagemap with shmem exceptions

[Peter Xu]

On Thu, Sep 23, 2021 at 06:46:18AM +0000, Tiberiu A Georgescu wrote:
> Mentioning the current missing information in the pagemap and alternatives
> on how to retrieve it, in case someone stumbles upon unexpected behaviour.
> 
> Signed-off-by: Tiberiu A Georgescu <tiberiu.georgescu@nutanix.com>
> Reviewed-by: Ivan Teterevkov <ivan.teterevkov@nutanix.com>
> Reviewed-by: Florian Schmidt <florian.schmidt@nutanix.com>
> Reviewed-by: Carl Waldspurger <carl.waldspurger@nutanix.com>
> Reviewed-by: Jonathan Davies <jonathan.davies@nutanix.com>

Reviewed-by: Peter Xu <peterx@redhat.com>

Thanks,

-- 
Peter Xu

Re: [PATCH v3 1/1] Documentation: update pagemap with shmem exceptions

[David Hildenbrand]

On 23.09.21 17:00, Peter Xu wrote:
> On Thu, Sep 23, 2021 at 06:46:18AM +0000, Tiberiu A Georgescu wrote:
>> Mentioning the current missing information in the pagemap and alternatives
>> on how to retrieve it, in case someone stumbles upon unexpected behaviour.
>>
>> Signed-off-by: Tiberiu A Georgescu <tiberiu.georgescu@nutanix.com>
>> Reviewed-by: Ivan Teterevkov <ivan.teterevkov@nutanix.com>
>> Reviewed-by: Florian Schmidt <florian.schmidt@nutanix.com>
>> Reviewed-by: Carl Waldspurger <carl.waldspurger@nutanix.com>
>> Reviewed-by: Jonathan Davies <jonathan.davies@nutanix.com>
> 
> Reviewed-by: Peter Xu <peterx@redhat.com>

LGTM as well; I like that we only give hints instead of listing steps to 
perform as in the previous version.

Reviewed-by: David Hildenbrand <david@redhat.com>


-- 
Thanks,

David / dhildenb

Re: [PATCH v3 0/1] Documenting shmem as an exception case for the pagemap

[Jonathan Corbet]

Tiberiu A Georgescu <tiberiu.georgescu@nutanix.com> writes:

> This patch follows the discussions on previous documentation patch threads
> [1][2]. It presents the exception case of shared memory management from the
> pagemap's point of view. It briefly describes what is missing, why it is
> missing and alternatives to the pagemap for page info retrieval in user
> space.
>
> In short, the kernel does not keep track of PTEs for swapped out shared
> pages within the processes that references them. Thus, the proc/pid/pagemap
> tool cannot print the swap destination of the shared memory pages, instead
> setting the pagemap entry to zero for both non-allocated and swapped out
> pages. This can create confusion for users who need information on swapped
> out pages.
>
> The reasons why maintaining the PTEs of all swapped out shared pages among
> all processes while maintaining similar performance is not a trivial task,
> or a desirable change, have been discussed extensively [1][3][4][5]. There
> are also arguments for why this arguably missing information should
> eventually be exposed to the user in either a future pagemap patch, or by
> an alternative tool.
>
> [1]: https://marc.info/?m=162878395426774
> [2]: https://lore.kernel.org/lkml/20210920164931.175411-1-tiberiu.georgescu@nutanix.com/
> [3]: https://lore.kernel.org/lkml/20210730160826.63785-1-tiberiu.georgescu@nutanix.com/
> [4]: https://lore.kernel.org/lkml/20210807032521.7591-1-peterx@redhat.com/
> [5]: https://lore.kernel.org/lkml/20210715201651.212134-1-peterx@redhat.com/
>
> Tiberiu A Georgescu (1):
>   Documentation: update pagemap with shmem exceptions
>
>  Documentation/admin-guide/mm/pagemap.rst | 22 ++++++++++++++++++++++
>  1 file changed, 22 insertions(+)

Applied, thanks.

jon