From: Daniel Colascione <firstname.lastname@example.org> To: Michal Hocko <email@example.com> Cc: Minchan Kim <firstname.lastname@example.org>, Andrew Morton <email@example.com>, LKML <firstname.lastname@example.org>, linux-mm <email@example.com>, Johannes Weiner <firstname.lastname@example.org>, Tim Murray <email@example.com>, Joel Fernandes <firstname.lastname@example.org>, Suren Baghdasaryan <email@example.com>, Shakeel Butt <firstname.lastname@example.org>, Sonny Rao <email@example.com>, Brian Geffon <firstname.lastname@example.org>, Linux API <email@example.com> Subject: Re: [RFC 7/7] mm: madvise support MADV_ANONYMOUS_FILTER and MADV_FILE_FILTER Date: Tue, 28 May 2019 02:39:03 -0700 [thread overview] Message-ID: <CAKOZueux3T4_dMOUK6R=ZHhCFaSSstOCPh_KSwSMCW_yp=jdSg@mail.gmail.com> (raw) In-Reply-To: <20190528090821.GU1658@dhcp22.suse.cz> On Tue, May 28, 2019 at 2:08 AM Michal Hocko <firstname.lastname@example.org> wrote: > > On Tue 28-05-19 17:49:27, Minchan Kim wrote: > > On Tue, May 28, 2019 at 01:31:13AM -0700, Daniel Colascione wrote: > > > On Tue, May 28, 2019 at 1:14 AM Minchan Kim <email@example.com> wrote: > > > > if we went with the per vma fd approach then you would get this > > > > > feature automatically because map_files would refer to file backed > > > > > mappings while map_anon could refer only to anonymous mappings. > > > > > > > > The reason to add such filter option is to avoid the parsing overhead > > > > so map_anon wouldn't be helpful. > > > > > > Without chiming on whether the filter option is a good idea, I'd like > > > to suggest that providing an efficient binary interfaces for pulling > > > memory map information out of processes. Some single-system-call > > > method for retrieving a binary snapshot of a process's address space > > > complete with attributes (selectable, like statx?) for each VMA would > > > reduce complexity and increase performance in a variety of areas, > > > e.g., Android memory map debugging commands. > > > > I agree it's the best we can get *generally*. > > Michal, any opinion? > > I am not really sure this is directly related. I think the primary > question that we have to sort out first is whether we want to have > the remote madvise call process or vma fd based. This is an important > distinction wrt. usability. I have only seen pid vs. pidfd discussions > so far unfortunately. I don't think the vma fd approach is viable. We have some processes with a *lot* of VMAs --- system_server had 4204 when I checked just now (and that's typical) --- and an FD operation per VMA would be excessive. VMAs also come and go pretty easily depending on changes in protections and various faults. It's also not entirely clear what the semantics of vma FDs should be over address space mutations, while the semantics of address ranges are well-understood. I would much prefer an interface operating on address ranges to one operating on VMA FDs, both for efficiency and for consistency with other memory management APIs. > An interface to query address range information is a separate but > although a related topic. We have /proc/<pid>/[s]maps for that right > now and I understand it is not a general win for all usecases because > it tends to be slow for some. I can see how /proc/<pid>/map_anons could > provide per vma information in a binary form via a fd based interface. > But I would rather not conflate those two discussions much - well except > if it could give one of the approaches more justification but let's > focus on the madvise part first. I don't think it's a good idea to focus on one feature in a multi-feature change when the interactions between features can be very important for overall design of the multi-feature system and the design of each feature. Here's my thinking on the high-level design: I'm imagining an address-range system that would work like this: we'd create some kind of process_vm_getinfo(2) system call  that would accept a statx-like attribute map and a pid/fd parameter as input and return, on output, two things: 1) an array  of VMA descriptors containing the requested information, and 2) a VMA configuration sequence number. We'd then have process_madvise() and other cross-process VM interfaces accept both address ranges and this sequence number; they'd succeed only if the VMA configuration sequence number is still current, i.e., the target process hasn't changed its VMA configuration (implicitly or explicitly) since the call to process_vm_getinfo(). This way, a process A that wants to perform some VM operation on process B can slurp B's VMA configuration using process_vm_getinfo(), figure out what it wants to do, and attempt to do it. If B modifies its memory map in the meantime, If A finds that its local knowledge of B's memory map has become invalid between the process_vm_getinfo() and A taking some action based on the result, A can retry . While A could instead ptrace or otherwise suspend B, *then* read B's memory map (knowing B is quiescent), *then* operate on B, the optimistic approach I'm describing would be much lighter-weight in the typical case. It's also pretty simple, IMHO. If the "operate on B" step is some kind of vectorized operation over multiple address ranges, this approach also gets us all-or-nothing semantics. Or maybe the whole sequence number thing is overkill and we don't need atomicity? But if there's a concern that A shouldn't operate on B's memory without knowing what it's operating on, then the scheme I've proposed above solves this knowledge problem in a pretty lightweight way.  or some other interface  or something more complicated if we want the descriptors to contain variable-length elements, e.g., strings  or override the sequence number check if it's feeling bold?
next prev parent reply other threads:[~2019-05-28 9:39 UTC|newest] Thread overview: 138+ messages / expand[flat|nested] mbox.gz Atom feed top 2019-05-20 3:52 [RFC 0/7] introduce memory hinting API for external process Minchan Kim 2019-05-20 3:52 ` [RFC 1/7] mm: introduce MADV_COOL Minchan Kim 2019-05-20 8:16 ` Michal Hocko 2019-05-20 8:19 ` Michal Hocko 2019-05-20 15:08 ` Suren Baghdasaryan 2019-05-20 22:55 ` Minchan Kim 2019-05-20 22:54 ` Minchan Kim 2019-05-21 6:04 ` Michal Hocko 2019-05-21 9:11 ` Minchan Kim 2019-05-21 10:05 ` Michal Hocko 2019-05-28 8:53 ` Hillf Danton 2019-05-28 10:58 ` Minchan Kim 2019-05-20 3:52 ` [RFC 2/7] mm: change PAGEREF_RECLAIM_CLEAN with PAGE_REFRECLAIM Minchan Kim 2019-05-20 16:50 ` Johannes Weiner 2019-05-20 22:57 ` Minchan Kim 2019-05-20 3:52 ` [RFC 3/7] mm: introduce MADV_COLD Minchan Kim 2019-05-20 8:27 ` Michal Hocko 2019-05-20 23:00 ` Minchan Kim 2019-05-21 6:08 ` Michal Hocko 2019-05-21 9:13 ` Minchan Kim 2019-05-28 14:54 ` Hillf Danton 2019-05-30 0:45 ` Minchan Kim 2019-05-20 3:52 ` [RFC 4/7] mm: factor out madvise's core functionality Minchan Kim 2019-05-20 14:26 ` Oleksandr Natalenko 2019-05-21 1:26 ` Minchan Kim 2019-05-21 6:36 ` Oleksandr Natalenko 2019-05-21 6:50 ` Michal Hocko 2019-05-21 7:06 ` Oleksandr Natalenko 2019-05-21 10:52 ` Minchan Kim 2019-05-21 11:00 ` Michal Hocko 2019-05-21 11:24 ` Minchan Kim 2019-05-21 11:32 ` Michal Hocko 2019-05-21 10:49 ` Minchan Kim 2019-05-21 10:55 ` Michal Hocko 2019-05-20 3:52 ` [RFC 5/7] mm: introduce external memory hinting API Minchan Kim 2019-05-20 9:18 ` Michal Hocko 2019-05-21 2:41 ` Minchan Kim 2019-05-21 6:17 ` Michal Hocko 2019-05-21 10:32 ` Minchan Kim 2019-05-21 9:01 ` Christian Brauner 2019-05-21 11:35 ` Minchan Kim 2019-05-21 11:51 ` Christian Brauner 2019-05-21 15:31 ` Oleg Nesterov 2019-05-27 7:43 ` Minchan Kim 2019-05-27 15:12 ` Oleg Nesterov 2019-05-27 23:33 ` Minchan Kim 2019-05-28 7:23 ` Michal Hocko 2019-05-29 3:41 ` Hillf Danton 2019-05-30 0:38 ` Minchan Kim 2019-05-20 3:52 ` [RFC 6/7] mm: extend process_madvise syscall to support vector arrary Minchan Kim 2019-05-20 9:22 ` Michal Hocko 2019-05-21 2:48 ` Minchan Kim 2019-05-21 6:24 ` Michal Hocko 2019-05-21 10:26 ` Minchan Kim 2019-05-21 10:37 ` Michal Hocko 2019-05-27 7:49 ` Minchan Kim 2019-05-29 10:08 ` Daniel Colascione 2019-05-29 10:33 ` Michal Hocko 2019-05-30 2:17 ` Minchan Kim 2019-05-30 6:57 ` Michal Hocko 2019-05-30 8:02 ` Minchan Kim 2019-05-30 16:19 ` Daniel Colascione 2019-05-30 18:47 ` Michal Hocko 2019-05-29 4:14 ` Hillf Danton 2019-05-30 0:35 ` Minchan Kim 2019-05-20 3:52 ` [RFC 7/7] mm: madvise support MADV_ANONYMOUS_FILTER and MADV_FILE_FILTER Minchan Kim 2019-05-20 9:28 ` Michal Hocko 2019-05-21 2:55 ` Minchan Kim 2019-05-21 6:26 ` Michal Hocko 2019-05-27 7:58 ` Minchan Kim 2019-05-27 12:44 ` Michal Hocko 2019-05-28 3:26 ` Minchan Kim 2019-05-28 6:29 ` Michal Hocko 2019-05-28 8:13 ` Minchan Kim 2019-05-28 8:31 ` Daniel Colascione 2019-05-28 8:49 ` Minchan Kim 2019-05-28 9:08 ` Michal Hocko 2019-05-28 9:39 ` Daniel Colascione [this message] 2019-05-28 10:33 ` Michal Hocko 2019-05-28 11:21 ` Daniel Colascione 2019-05-28 11:49 ` Michal Hocko 2019-05-28 12:11 ` Daniel Colascione 2019-05-28 12:32 ` Michal Hocko 2019-05-28 10:32 ` Minchan Kim 2019-05-28 10:41 ` Michal Hocko 2019-05-28 11:12 ` Minchan Kim 2019-05-28 11:28 ` Michal Hocko 2019-05-28 11:42 ` Daniel Colascione 2019-05-28 11:56 ` Michal Hocko 2019-05-28 12:18 ` Daniel Colascione 2019-05-28 12:38 ` Michal Hocko 2019-05-28 12:10 ` Minchan Kim 2019-05-28 11:44 ` Minchan Kim 2019-05-28 11:51 ` Daniel Colascione 2019-05-28 12:06 ` Michal Hocko 2019-05-28 12:22 ` Minchan Kim 2019-05-28 11:28 ` Daniel Colascione 2019-05-21 15:33 ` Johannes Weiner 2019-05-22 1:50 ` Minchan Kim 2019-05-29 4:36 ` Hillf Danton 2019-05-30 1:00 ` Minchan Kim 2019-05-20 6:37 ` [RFC 0/7] introduce memory hinting API for external process Anshuman Khandual 2019-05-20 16:59 ` Tim Murray 2019-05-21 2:55 ` Anshuman Khandual 2019-05-21 5:14 ` Minchan Kim 2019-05-21 10:34 ` Michal Hocko 2019-05-28 10:50 ` Anshuman Khandual 2019-05-21 12:56 ` Shakeel Butt 2019-05-22 4:15 ` Brian Geffon 2019-05-22 4:23 ` Brian Geffon 2019-05-20 9:28 ` Michal Hocko 2019-05-20 14:42 ` Oleksandr Natalenko 2019-05-21 2:56 ` Minchan Kim 2019-05-20 16:46 ` Johannes Weiner 2019-05-21 4:39 ` Minchan Kim 2019-05-21 6:32 ` Michal Hocko 2019-05-21 1:44 ` Matthew Wilcox 2019-05-21 5:01 ` Minchan Kim 2019-05-21 6:34 ` Michal Hocko 2019-05-21 8:42 ` Christian Brauner 2019-05-21 11:05 ` Minchan Kim 2019-05-21 11:30 ` Christian Brauner 2019-05-21 11:39 ` Christian Brauner 2019-05-22 5:11 ` Daniel Colascione 2019-05-22 8:22 ` Christian Brauner 2019-05-22 13:16 ` Daniel Colascione 2019-05-22 14:52 ` Christian Brauner 2019-05-22 15:17 ` Daniel Colascione 2019-05-22 15:48 ` Christian Brauner 2019-05-22 15:57 ` Daniel Colascione 2019-05-22 16:01 ` Christian Brauner 2019-05-22 16:01 ` Daniel Colascione 2019-05-23 13:07 ` Minchan Kim 2019-05-27 8:06 ` Minchan Kim 2019-05-21 11:41 ` Minchan Kim 2019-05-21 12:04 ` Christian Brauner 2019-05-21 12:15 ` Oleksandr Natalenko 2019-05-21 12:53 ` Shakeel Butt
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to='CAKOZueux3T4_dMOUK6R=ZHhCFaSSstOCPh_KSwSMCW_yp=jdSg@mail.gmail.com' \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --subject='Re: [RFC 7/7] mm: madvise support MADV_ANONYMOUS_FILTER and MADV_FILE_FILTER' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).