From: Michal Hocko <mhocko@kernel.org> To: "Kirill A. Shutemov" <kirill@shutemov.name> Cc: Kirill Tkhai <ktkhai@virtuozzo.com>, Minchan Kim <minchan@kernel.org>, Andrew Morton <akpm@linux-foundation.org>, LKML <linux-kernel@vger.kernel.org>, linux-mm <linux-mm@kvack.org>, linux-api@vger.kernel.org, oleksandr@redhat.com, Suren Baghdasaryan <surenb@google.com>, Tim Murray <timmurray@google.com>, Daniel Colascione <dancol@google.com>, Sandeep Patil <sspatil@google.com>, Sonny Rao <sonnyrao@google.com>, Brian Geffon <bgeffon@google.com>, Johannes Weiner <hannes@cmpxchg.org>, Shakeel Butt <shakeelb@google.com>, John Dias <joaodias@google.com>, christian.brauner@ubuntu.com, sjpark@amazon.de Subject: Re: [PATCH v2 2/5] mm: introduce external memory hinting API Date: Mon, 20 Jan 2020 14:24:05 +0100 [thread overview] Message-ID: <20200120132405.GF18451@dhcp22.suse.cz> (raw) In-Reply-To: <20200120123935.onlls7enjtzenbvt@box> On Mon 20-01-20 15:39:35, Kirill A. Shutemov wrote: > On Mon, Jan 20, 2020 at 12:27:22PM +0100, Michal Hocko wrote: > > On Mon 20-01-20 13:24:35, Kirill Tkhai wrote: [...] > > > Even two threads on common memory need a synchronization > > > to manage mappings in a sane way. Managing memory from two processes > > > is the same in principle, and the only difference is that another level > > > of synchronization is required. > > > > Well, not really. The operation might simply attempt to perform an > > operation on a specific memory area and get a failure if it doesn't > > reference the same object anymore. What I think we need is some form of > > a handle to operate on. In the past we have discussed several > > directions. I was proposing /proc/self/map_anon/ (analogous to > > map_files) where you could inspect anonymous memory and get a file > > handle for it. madvise would then operate on the fd and then there > > shouldn't be a real problem to revalidate that the object is still > > valid. But there was no general enthusiasm about that approach. There > > are likely some land mines on the way. > > Converting anon memory to file-backed is bad idea and going to backfire. I didn't mean to convert. I meant to expose that information via proc the same way we do for file backed mappings. That shouldn't really require to re-design the way how anonymous vma work IMO. But I haven't tried that so there might be many gotchas there. There are obvious things to think about though. Such fd cannot be sent to other processes (SCM stuff), mmap of the file would have to be disallowed and many others I am not aware of. I am not even pushing this direction because I am not convinced about how viable it is myself. But it would sound like a nice extension of the existing mechanism we have and a file based madvise sounds attractive to me as well because we already have that. -- Michal Hocko SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> To: "Kirill A. Shutemov" <kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org> Cc: Kirill Tkhai <ktkhai-5HdwGun5lf+gSpxsJD1C4w@public.gmane.org>, Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, Andrew Morton <akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>, LKML <linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, linux-mm <linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org>, linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, oleksandr-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, Suren Baghdasaryan <surenb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Tim Murray <timmurray-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Daniel Colascione <dancol-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Sandeep Patil <sspatil-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Sonny Rao <sonnyrao-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Brian Geffon <bgeffon-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>, Shakeel Butt <shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, John Dias <joaodias-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>, christian.brauner-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org, sjpark-ebkRAfMGSJGzQB+pC5nmwQ@public.gmane.org Subject: Re: [PATCH v2 2/5] mm: introduce external memory hinting API Date: Mon, 20 Jan 2020 14:24:05 +0100 [thread overview] Message-ID: <20200120132405.GF18451@dhcp22.suse.cz> (raw) In-Reply-To: <20200120123935.onlls7enjtzenbvt@box> On Mon 20-01-20 15:39:35, Kirill A. Shutemov wrote: > On Mon, Jan 20, 2020 at 12:27:22PM +0100, Michal Hocko wrote: > > On Mon 20-01-20 13:24:35, Kirill Tkhai wrote: [...] > > > Even two threads on common memory need a synchronization > > > to manage mappings in a sane way. Managing memory from two processes > > > is the same in principle, and the only difference is that another level > > > of synchronization is required. > > > > Well, not really. The operation might simply attempt to perform an > > operation on a specific memory area and get a failure if it doesn't > > reference the same object anymore. What I think we need is some form of > > a handle to operate on. In the past we have discussed several > > directions. I was proposing /proc/self/map_anon/ (analogous to > > map_files) where you could inspect anonymous memory and get a file > > handle for it. madvise would then operate on the fd and then there > > shouldn't be a real problem to revalidate that the object is still > > valid. But there was no general enthusiasm about that approach. There > > are likely some land mines on the way. > > Converting anon memory to file-backed is bad idea and going to backfire. I didn't mean to convert. I meant to expose that information via proc the same way we do for file backed mappings. That shouldn't really require to re-design the way how anonymous vma work IMO. But I haven't tried that so there might be many gotchas there. There are obvious things to think about though. Such fd cannot be sent to other processes (SCM stuff), mmap of the file would have to be disallowed and many others I am not aware of. I am not even pushing this direction because I am not convinced about how viable it is myself. But it would sound like a nice extension of the existing mechanism we have and a file based madvise sounds attractive to me as well because we already have that. -- Michal Hocko SUSE Labs
next prev parent reply other threads:[~2020-01-20 13:24 UTC|newest] Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-16 23:59 [PATCH v2 0/5] introduce memory hinting API for external process Minchan Kim 2020-01-16 23:59 ` [PATCH v2 1/5] mm: factor out madvise's core functionality Minchan Kim 2020-01-17 10:02 ` Kirill Tkhai 2020-01-17 10:02 ` Kirill Tkhai 2020-01-17 18:14 ` Minchan Kim 2020-01-17 18:14 ` Minchan Kim 2020-01-16 23:59 ` [PATCH v2 2/5] mm: introduce external memory hinting API Minchan Kim 2020-01-16 23:59 ` Minchan Kim 2020-01-17 11:52 ` Michal Hocko 2020-01-17 11:52 ` Michal Hocko 2020-01-17 15:58 ` Kirill A. Shutemov 2020-01-17 15:58 ` Kirill A. Shutemov 2020-01-17 17:32 ` Minchan Kim 2020-01-17 17:32 ` Minchan Kim 2020-01-17 21:26 ` Kirill A. Shutemov 2020-01-17 21:26 ` Kirill A. Shutemov 2020-01-18 9:40 ` SeongJae Park 2020-01-18 9:40 ` SeongJae Park 2020-01-19 16:14 ` sspatil 2020-01-19 16:14 ` sspatil-hpIqsD4AKlfQT0dZR+AlfA 2020-01-20 7:58 ` Michal Hocko 2020-01-20 10:39 ` Kirill Tkhai 2020-01-20 10:39 ` Kirill Tkhai 2020-01-21 18:32 ` Minchan Kim 2020-01-22 8:28 ` Michal Hocko 2020-01-22 8:28 ` Michal Hocko 2020-01-22 9:36 ` SeongJae Park 2020-01-22 9:36 ` SeongJae Park 2020-01-22 10:02 ` Michal Hocko 2020-01-22 10:02 ` Michal Hocko 2020-01-22 13:28 ` SeongJae Park 2020-01-22 13:28 ` SeongJae Park 2020-01-23 1:41 ` Minchan Kim 2020-01-23 1:41 ` Minchan Kim 2020-01-23 9:13 ` Michal Hocko 2020-01-21 18:11 ` Minchan Kim 2020-01-21 18:11 ` Minchan Kim 2020-01-22 10:44 ` Oleksandr Natalenko 2020-01-23 1:43 ` Minchan Kim 2020-01-23 7:29 ` Oleksandr Natalenko 2020-01-17 17:25 ` Minchan Kim 2020-01-17 17:25 ` Minchan Kim 2020-01-20 8:03 ` Michal Hocko 2020-01-20 8:03 ` Michal Hocko 2020-01-20 10:24 ` Kirill Tkhai 2020-01-20 10:24 ` Kirill Tkhai 2020-01-20 11:27 ` Michal Hocko 2020-01-20 11:27 ` Michal Hocko 2020-01-20 12:39 ` Kirill A. Shutemov 2020-01-20 13:24 ` Michal Hocko [this message] 2020-01-20 13:24 ` Michal Hocko 2020-01-20 14:21 ` Kirill A. Shutemov 2020-01-20 15:44 ` Michal Hocko 2020-01-20 15:44 ` Michal Hocko 2020-01-21 18:43 ` Minchan Kim 2020-01-21 18:43 ` Minchan Kim 2020-01-16 23:59 ` [PATCH v2 3/5] mm/madvise: employ mmget_still_valid for write lock Minchan Kim 2020-01-16 23:59 ` [PATCH v2 4/5] mm/madvise: allow KSM hints for remote API Minchan Kim 2020-01-17 10:13 ` Kirill Tkhai 2020-01-17 10:13 ` Kirill Tkhai 2020-01-17 12:34 ` Oleksandr Natalenko 2020-01-17 12:34 ` Oleksandr Natalenko 2020-01-21 17:45 ` Minchan Kim 2020-01-21 17:45 ` Minchan Kim 2020-01-16 23:59 ` [PATCH v2 5/5] mm: support both pid and pidfd for process_madvise Minchan Kim
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200120132405.GF18451@dhcp22.suse.cz \ --to=mhocko@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=bgeffon@google.com \ --cc=christian.brauner@ubuntu.com \ --cc=dancol@google.com \ --cc=hannes@cmpxchg.org \ --cc=joaodias@google.com \ --cc=kirill@shutemov.name \ --cc=ktkhai@virtuozzo.com \ --cc=linux-api@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=minchan@kernel.org \ --cc=oleksandr@redhat.com \ --cc=shakeelb@google.com \ --cc=sjpark@amazon.de \ --cc=sonnyrao@google.com \ --cc=sspatil@google.com \ --cc=surenb@google.com \ --cc=timmurray@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.