From: Andrew Morton <akpm@linux-foundation.org>
To: Suren Baghdasaryan <surenb@google.com>
Cc: ccross@google.com, sumit.semwal@linaro.org, mhocko@suse.com,
dave.hansen@intel.com, keescook@chromium.org,
willy@infradead.org, kirill.shutemov@linux.intel.com,
vbabka@suse.cz, hannes@cmpxchg.org, corbet@lwn.net,
viro@zeniv.linux.org.uk, rdunlap@infradead.org,
kaleshsingh@google.com, peterx@redhat.com, rppt@kernel.org,
peterz@infradead.org, catalin.marinas@arm.com,
vincenzo.frascino@arm.com, chinwen.chang@mediatek.com,
axelrasmussen@google.com, aarcange@redhat.com, jannh@google.com,
apopple@nvidia.com, jhubbard@nvidia.com, yuzhao@google.com,
will@kernel.org, fenghua.yu@intel.com,
thunder.leizhen@huawei.com, hughd@google.com,
feng.tang@intel.com, jgg@ziepe.ca, guro@fb.com,
tglx@linutronix.de, krisman@collabora.com,
chris.hyser@oracle.com, pcc@google.com, ebiederm@xmission.com,
axboe@kernel.dk, legion@kernel.org, eb@emlix.com,
gorcunov@gmail.com, pavel@ucw.cz, songmuchun@bytedance.com,
viresh.kumar@linaro.org, thomascedeno@google.com,
sashal@kernel.org, cxfcosmos@gmail.com, linux@rasmusvillemoes.dk,
linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
linux-doc@vger.kernel.org, linux-mm@kvack.org,
kernel-team@android.com
Subject: Re: [PATCH v10 2/3] mm: add a field to store names for private anonymous memory
Date: Fri, 1 Oct 2021 16:08:30 -0700 [thread overview]
Message-ID: <20211001160830.700c36b32b736478000b3420@linux-foundation.org> (raw)
In-Reply-To: <20211001205657.815551-2-surenb@google.com>
On Fri, 1 Oct 2021 13:56:56 -0700 Suren Baghdasaryan <surenb@google.com> wrote:
> From: Colin Cross <ccross@google.com>
>
> In many userspace applications, and especially in VM based applications
> like Android uses heavily, there are multiple different allocators in use.
> At a minimum there is libc malloc and the stack, and in many cases there
> are libc malloc, the stack, direct syscalls to mmap anonymous memory, and
> multiple VM heaps (one for small objects, one for big objects, etc.).
> Each of these layers usually has its own tools to inspect its usage;
> malloc by compiling a debug version, the VM through heap inspection tools,
> and for direct syscalls there is usually no way to track them.
>
> On Android we heavily use a set of tools that use an extended version of
> the logic covered in Documentation/vm/pagemap.txt to walk all pages mapped
> in userspace and slice their usage by process, shared (COW) vs. unique
> mappings, backing, etc. This can account for real physical memory usage
> even in cases like fork without exec (which Android uses heavily to share
> as many private COW pages as possible between processes), Kernel SamePage
> Merging, and clean zero pages. It produces a measurement of the pages
> that only exist in that process (USS, for unique), and a measurement of
> the physical memory usage of that process with the cost of shared pages
> being evenly split between processes that share them (PSS).
>
> If all anonymous memory is indistinguishable then figuring out the real
> physical memory usage (PSS) of each heap requires either a pagemap walking
> tool that can understand the heap debugging of every layer, or for every
> layer's heap debugging tools to implement the pagemap walking logic, in
> which case it is hard to get a consistent view of memory across the whole
> system.
>
> Tracking the information in userspace leads to all sorts of problems.
> It either needs to be stored inside the process, which means every
> process has to have an API to export its current heap information upon
> request, or it has to be stored externally in a filesystem that
> somebody needs to clean up on crashes. It needs to be readable while
> the process is still running, so it has to have some sort of
> synchronization with every layer of userspace. Efficiently tracking
> the ranges requires reimplementing something like the kernel vma
> trees, and linking to it from every layer of userspace. It requires
> more memory, more syscalls, more runtime cost, and more complexity to
> separately track regions that the kernel is already tracking.
>
> This patch adds a field to /proc/pid/maps and /proc/pid/smaps to show a
> userspace-provided name for anonymous vmas. The names of named anonymous
> vmas are shown in /proc/pid/maps and /proc/pid/smaps as [anon:<name>].
>
> Userspace can set the name for a region of memory by calling
> prctl(PR_SET_VMA, PR_SET_VMA_ANON_NAME, start, len, (unsigned long)name);
So this can cause a vma to be split, if [start,len] doesn't exactly
describe an existing vma? If so, is this at all useful? If not then
`len' isn't needed - just pass in some address within an existing vma?
> Setting the name to NULL clears it. The name length limit is 80 bytes
> including NUL-terminator and is checked to contain only printable ascii
> characters (including space), except '[',']','\','$' and '`'.
>
> The name is stored in a pointer in the shared union in vm_area_struct
> that points to a null terminated string. Anonymous vmas with the same
> name (equivalent strings) and are otherwise mergeable will be merged.
So this can prevent vma merging if used incorrectly (or maliciously -
can't think how)? What are the potential impacts of this?
> The name pointers are not shared between vmas even if they contain the
> same name. The name pointer is stored in a union with fields that are
> only used on file-backed mappings, so it does not increase memory usage.
>
> The patch is based on the original patch developed by Colin Cross, more
> specifically on its latest version [1] posted upstream by Sumit Semwal.
> It used a userspace pointer to store vma names. In that design, name
> pointers could be shared between vmas. However during the last upstreaming
> attempt, Kees Cook raised concerns [2] about this approach and suggested
> to copy the name into kernel memory space, perform validity checks [3]
> and store as a string referenced from vm_area_struct.
> One big concern is about fork() performance which would need to strdup
> anonymous vma names. Dave Hansen suggested experimenting with worst-case
> scenario of forking a process with 64k vmas having longest possible names
> [4]. I ran this experiment on an ARM64 Android device and recorded a
> worst-case regression of almost 40% when forking such a process. This
> regression is addressed in the followup patch which replaces the pointer
> to a name with a refcounted structure that allows sharing the name pointer
> between vmas of the same name. Instead of duplicating the string during
> fork() or when splitting a vma it increments the refcount.
Generally, the patch adds a bunch of code which a lot of users won't
want. Did we bust a gut to reduce this impact? Was a standalone
config setting considered?
And what would be the impact of making this feature optional? Is a
proliferation of formats in /proc/pid/maps going to make userspace
parsers harder to develop and test?
I do think that saying "The names of named anonymous vmas are shown in
/proc/pid/maps and /proc/pid/smaps as [anon:<name>]." is a bit thin.
Please provide sample output so we can consider these things better.
What are the risks that existing parsers will be broken by such changes?
next prev parent reply other threads:[~2021-10-01 23:08 UTC|newest]
Thread overview: 80+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-01 20:56 [PATCH v10 1/3] mm: rearrange madvise code to allow for reuse Suren Baghdasaryan
2021-10-01 20:56 ` [PATCH v10 2/3] mm: add a field to store names for private anonymous memory Suren Baghdasaryan
2021-10-01 23:08 ` Andrew Morton [this message]
2021-10-02 0:52 ` Suren Baghdasaryan
2021-10-04 16:21 ` Suren Baghdasaryan
2021-10-07 2:39 ` Andrew Morton
2021-10-07 2:50 ` Suren Baghdasaryan
2021-10-01 20:56 ` [PATCH v10 3/3] mm: add anonymous vma name refcounting Suren Baghdasaryan
2021-10-05 18:42 ` Pavel Machek
2021-10-05 19:14 ` Suren Baghdasaryan
2021-10-05 19:21 ` Kees Cook
2021-10-05 20:04 ` Pavel Machek
2021-10-05 20:43 ` Suren Baghdasaryan
2021-10-06 6:57 ` John Hubbard
2021-10-06 8:27 ` Michal Hocko
2021-10-06 9:27 ` David Hildenbrand
2021-10-06 15:01 ` Suren Baghdasaryan
2021-10-06 15:07 ` David Hildenbrand
2021-10-06 15:20 ` Suren Baghdasaryan
2021-10-07 2:29 ` Andrew Morton
2021-10-07 2:46 ` Suren Baghdasaryan
2021-10-07 2:53 ` Andrew Morton
2021-10-07 3:01 ` Suren Baghdasaryan
2021-10-07 7:27 ` David Hildenbrand
2021-10-07 7:33 ` David Hildenbrand
2021-10-07 15:42 ` Suren Baghdasaryan
2021-10-06 17:58 ` Pavel Machek
2021-10-06 18:18 ` Suren Baghdasaryan
2021-10-07 8:10 ` Michal Hocko
2021-10-07 8:41 ` Pavel Machek
2021-10-07 8:47 ` Rasmus Villemoes
2021-10-07 10:15 ` Pavel Machek
2021-10-07 16:04 ` Suren Baghdasaryan
2021-10-07 16:40 ` Michal Hocko
2021-10-07 16:58 ` Suren Baghdasaryan
2021-10-07 17:31 ` Michal Hocko
2021-10-07 17:50 ` Suren Baghdasaryan
2021-10-07 18:12 ` Kees Cook
2021-10-07 18:50 ` Suren Baghdasaryan
2021-10-07 19:02 ` John Hubbard
2021-10-07 21:32 ` Suren Baghdasaryan
2021-10-08 1:04 ` Liam Howlett
2021-10-08 7:25 ` Rasmus Villemoes
2021-10-08 7:43 ` David Hildenbrand
2021-10-08 21:13 ` Kees Cook
2021-10-08 6:34 ` Michal Hocko
2021-10-08 14:14 ` Dave Hansen
2021-10-08 14:57 ` Michal Hocko
2021-10-08 16:10 ` Suren Baghdasaryan
2021-10-08 20:58 ` Kees Cook
2021-10-11 8:36 ` Michal Hocko
2021-10-12 1:18 ` Suren Baghdasaryan
2021-10-12 1:20 ` Suren Baghdasaryan
2021-10-12 3:00 ` Johannes Weiner
2021-10-12 5:36 ` Suren Baghdasaryan
2021-10-12 18:26 ` Johannes Weiner
2021-10-12 18:52 ` Suren Baghdasaryan
2021-10-12 20:41 ` Johannes Weiner
2021-10-12 20:59 ` Suren Baghdasaryan
2021-10-12 7:36 ` Michal Hocko
2021-10-12 16:50 ` Suren Baghdasaryan
2021-10-12 7:43 ` David Hildenbrand
2021-10-12 17:01 ` Suren Baghdasaryan
2021-10-14 20:16 ` Suren Baghdasaryan
2021-10-15 8:03 ` David Hildenbrand
2021-10-15 16:30 ` Suren Baghdasaryan
2021-10-15 16:39 ` David Hildenbrand
2021-10-15 18:33 ` Suren Baghdasaryan
2021-10-15 17:45 ` Kees Cook
2021-10-07 7:59 ` Michal Hocko
2021-10-07 15:45 ` Suren Baghdasaryan
2021-10-07 16:37 ` Michal Hocko
2021-10-07 16:43 ` Suren Baghdasaryan
2021-10-07 17:25 ` Michal Hocko
2021-10-07 17:30 ` Suren Baghdasaryan
2021-10-04 7:03 ` [PATCH v10 1/3] mm: rearrange madvise code to allow for reuse Rolf Eike Beer
2021-10-04 16:18 ` Suren Baghdasaryan
2021-10-05 21:00 ` Liam Howlett
2021-10-05 21:30 ` Suren Baghdasaryan
2021-10-06 17:33 ` Liam Howlett
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211001160830.700c36b32b736478000b3420@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=aarcange@redhat.com \
--cc=apopple@nvidia.com \
--cc=axboe@kernel.dk \
--cc=axelrasmussen@google.com \
--cc=catalin.marinas@arm.com \
--cc=ccross@google.com \
--cc=chinwen.chang@mediatek.com \
--cc=chris.hyser@oracle.com \
--cc=corbet@lwn.net \
--cc=cxfcosmos@gmail.com \
--cc=dave.hansen@intel.com \
--cc=eb@emlix.com \
--cc=ebiederm@xmission.com \
--cc=feng.tang@intel.com \
--cc=fenghua.yu@intel.com \
--cc=gorcunov@gmail.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=jannh@google.com \
--cc=jgg@ziepe.ca \
--cc=jhubbard@nvidia.com \
--cc=kaleshsingh@google.com \
--cc=keescook@chromium.org \
--cc=kernel-team@android.com \
--cc=kirill.shutemov@linux.intel.com \
--cc=krisman@collabora.com \
--cc=legion@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=linux@rasmusvillemoes.dk \
--cc=mhocko@suse.com \
--cc=pavel@ucw.cz \
--cc=pcc@google.com \
--cc=peterx@redhat.com \
--cc=peterz@infradead.org \
--cc=rdunlap@infradead.org \
--cc=rppt@kernel.org \
--cc=sashal@kernel.org \
--cc=songmuchun@bytedance.com \
--cc=sumit.semwal@linaro.org \
--cc=surenb@google.com \
--cc=tglx@linutronix.de \
--cc=thomascedeno@google.com \
--cc=thunder.leizhen@huawei.com \
--cc=vbabka@suse.cz \
--cc=vincenzo.frascino@arm.com \
--cc=viresh.kumar@linaro.org \
--cc=viro@zeniv.linux.org.uk \
--cc=will@kernel.org \
--cc=willy@infradead.org \
--cc=yuzhao@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).