From: Michal Hocko <mhocko@kernel.org> To: SeongJae Park <sjpark@amazon.com> Cc: Minchan Kim <minchan@kernel.org>, sspatil@google.com, kirill@shutemov.name, akpm@linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-api@vger.kernel.org, oleksandr@redhat.com, surenb@google.com, timmurray@google.com, dancol@google.com, sonnyrao@google.com, bgeffon@google.com, hannes@cmpxchg.org, shakeelb@google.com, joaodias@google.com, ktkhai@virtuozzo.com, christian.brauner@ubuntu.com, sjpark@amazon.de Subject: Re: Re: [PATCH v2 2/5] mm: introduce external memory hinting API Date: Wed, 22 Jan 2020 11:02:33 +0100 [thread overview] Message-ID: <20200122100233.GT29276@dhcp22.suse.cz> (raw) In-Reply-To: <20200122093624.14799-1-sjpark@amazon.com> On Wed 22-01-20 10:36:24, SeongJae Park wrote: > On Wed, 22 Jan 2020 09:28:53 +0100 Michal Hocko <mhocko@kernel.org> wrote: > > > On Tue 21-01-20 10:32:12, Minchan Kim wrote: > > > On Mon, Jan 20, 2020 at 08:58:25AM +0100, Michal Hocko wrote: > > [...] > > > > The interface really has to be robust to future potential usecases. > > > > > > I do understand your concern but for me, it's chicken and egg problem. > > > We usually do best effort to make something perfect as far as possible > > > but we also don't do over-engineering without real usecase from the > > > beginning. > > > > > > I already told you how we could synchronize among processes and potential > > > way to be extended Daniel suggested(That's why current API has extra field > > > for the cookie) even though we don't need it right now. > > > > If you can synchronize with the target task then you do not need a > > remote interface. Just use ptrace and you are done with it. > > > > > If you want to suggest the other way, please explain why your idea is > > > better and why we need it at this moment. > > > > I believe I have explained my concerns and why they matter. All you are > > saying is that you do not care because your particular usecase doesn't > > care. And that is a first signal of a future disaster when we end up > > with a broken and unfixable interface we have to maintain for ever. > > > > I will not go as far as to nack this but you should seriously think > > about other potential usecases and how they would work and what we are > > going to do when a first non-cooperative userspace memory management > > usecase materializes. > > Beside of the specific environment of Android, I think there are many ways to > know the address space layout and access patterns of other processes. The > idle_page_tracking might be an example that widelay available. > > Of course, the information might not strictly correct due to the timing issue, > but could be still worth to be used under some extreme situations, such as > memory pressure or fragmentation. For the same reason, ptrace() would not be > sufficient, as we have no perfect control, but only some level of control that > would be useful under specific situations. I am not sure I see your point. I am talking about races where a remote task is operating on a completely different object because the one it checked for has been unmapped and new one mapped over it. Memory pressure or a fragmentation will not change the object itself. Sure the memory might be reclaimed but that should be completely OK unless I am missing something. > I assume the users of this systemcall would understand the tradeoff and make > decisions. I disagree. My experience tells me that users tend to squeeze the maximum and beyond and hope they get what they want. > Also, as the users already have the right to do the tradeoff, I > think it's fair. In other words, I think the caller has both the power and the > responsibility to deal with the time-to-check-time-to-react problem. > > Nonetheless, I also agree this is important concern and the patch would be > better if it adds more detailed documentation regarding this issue. If there is _really_ a strong consensus that the racy interface is reasonable then it absolutely has to be described with a clearly state that those races might result in hard to predict behavior unless all tasks sharing the address space are blocked between the check and the madvise call. -- Michal Hocko SUSE Labs
WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> To: SeongJae Park <sjpark-vV1OtcyAfmbQT0dZR+AlfA@public.gmane.org> Cc: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>, sspatil-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org, akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org, linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org, linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, oleksandr-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org, surenb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, timmurray-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, dancol-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, sonnyrao-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, bgeffon-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org, shakeelb-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, joaodias-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org, ktkhai-5HdwGun5lf+gSpxsJD1C4w@public.gmane.org, christian.brauner-GeWIH/nMZzLQT0dZR+AlfA@public.gmane.org, sjpark-ebkRAfMGSJGzQB+pC5nmwQ@public.gmane.org Subject: Re: Re: [PATCH v2 2/5] mm: introduce external memory hinting API Date: Wed, 22 Jan 2020 11:02:33 +0100 [thread overview] Message-ID: <20200122100233.GT29276@dhcp22.suse.cz> (raw) In-Reply-To: <20200122093624.14799-1-sjpark-vV1OtcyAfmbQT0dZR+AlfA@public.gmane.org> On Wed 22-01-20 10:36:24, SeongJae Park wrote: > On Wed, 22 Jan 2020 09:28:53 +0100 Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote: > > > On Tue 21-01-20 10:32:12, Minchan Kim wrote: > > > On Mon, Jan 20, 2020 at 08:58:25AM +0100, Michal Hocko wrote: > > [...] > > > > The interface really has to be robust to future potential usecases. > > > > > > I do understand your concern but for me, it's chicken and egg problem. > > > We usually do best effort to make something perfect as far as possible > > > but we also don't do over-engineering without real usecase from the > > > beginning. > > > > > > I already told you how we could synchronize among processes and potential > > > way to be extended Daniel suggested(That's why current API has extra field > > > for the cookie) even though we don't need it right now. > > > > If you can synchronize with the target task then you do not need a > > remote interface. Just use ptrace and you are done with it. > > > > > If you want to suggest the other way, please explain why your idea is > > > better and why we need it at this moment. > > > > I believe I have explained my concerns and why they matter. All you are > > saying is that you do not care because your particular usecase doesn't > > care. And that is a first signal of a future disaster when we end up > > with a broken and unfixable interface we have to maintain for ever. > > > > I will not go as far as to nack this but you should seriously think > > about other potential usecases and how they would work and what we are > > going to do when a first non-cooperative userspace memory management > > usecase materializes. > > Beside of the specific environment of Android, I think there are many ways to > know the address space layout and access patterns of other processes. The > idle_page_tracking might be an example that widelay available. > > Of course, the information might not strictly correct due to the timing issue, > but could be still worth to be used under some extreme situations, such as > memory pressure or fragmentation. For the same reason, ptrace() would not be > sufficient, as we have no perfect control, but only some level of control that > would be useful under specific situations. I am not sure I see your point. I am talking about races where a remote task is operating on a completely different object because the one it checked for has been unmapped and new one mapped over it. Memory pressure or a fragmentation will not change the object itself. Sure the memory might be reclaimed but that should be completely OK unless I am missing something. > I assume the users of this systemcall would understand the tradeoff and make > decisions. I disagree. My experience tells me that users tend to squeeze the maximum and beyond and hope they get what they want. > Also, as the users already have the right to do the tradeoff, I > think it's fair. In other words, I think the caller has both the power and the > responsibility to deal with the time-to-check-time-to-react problem. > > Nonetheless, I also agree this is important concern and the patch would be > better if it adds more detailed documentation regarding this issue. If there is _really_ a strong consensus that the racy interface is reasonable then it absolutely has to be described with a clearly state that those races might result in hard to predict behavior unless all tasks sharing the address space are blocked between the check and the madvise call. -- Michal Hocko SUSE Labs
next prev parent reply other threads:[~2020-01-22 10:02 UTC|newest] Thread overview: 65+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-16 23:59 [PATCH v2 0/5] introduce memory hinting API for external process Minchan Kim 2020-01-16 23:59 ` [PATCH v2 1/5] mm: factor out madvise's core functionality Minchan Kim 2020-01-17 10:02 ` Kirill Tkhai 2020-01-17 10:02 ` Kirill Tkhai 2020-01-17 18:14 ` Minchan Kim 2020-01-17 18:14 ` Minchan Kim 2020-01-16 23:59 ` [PATCH v2 2/5] mm: introduce external memory hinting API Minchan Kim 2020-01-16 23:59 ` Minchan Kim 2020-01-17 11:52 ` Michal Hocko 2020-01-17 11:52 ` Michal Hocko 2020-01-17 15:58 ` Kirill A. Shutemov 2020-01-17 15:58 ` Kirill A. Shutemov 2020-01-17 17:32 ` Minchan Kim 2020-01-17 17:32 ` Minchan Kim 2020-01-17 21:26 ` Kirill A. Shutemov 2020-01-17 21:26 ` Kirill A. Shutemov 2020-01-18 9:40 ` SeongJae Park 2020-01-18 9:40 ` SeongJae Park 2020-01-19 16:14 ` sspatil 2020-01-19 16:14 ` sspatil-hpIqsD4AKlfQT0dZR+AlfA 2020-01-20 7:58 ` Michal Hocko 2020-01-20 10:39 ` Kirill Tkhai 2020-01-20 10:39 ` Kirill Tkhai 2020-01-21 18:32 ` Minchan Kim 2020-01-22 8:28 ` Michal Hocko 2020-01-22 8:28 ` Michal Hocko 2020-01-22 9:36 ` SeongJae Park 2020-01-22 9:36 ` SeongJae Park 2020-01-22 10:02 ` Michal Hocko [this message] 2020-01-22 10:02 ` Michal Hocko 2020-01-22 13:28 ` SeongJae Park 2020-01-22 13:28 ` SeongJae Park 2020-01-23 1:41 ` Minchan Kim 2020-01-23 1:41 ` Minchan Kim 2020-01-23 9:13 ` Michal Hocko 2020-01-21 18:11 ` Minchan Kim 2020-01-21 18:11 ` Minchan Kim 2020-01-22 10:44 ` Oleksandr Natalenko 2020-01-23 1:43 ` Minchan Kim 2020-01-23 7:29 ` Oleksandr Natalenko 2020-01-17 17:25 ` Minchan Kim 2020-01-17 17:25 ` Minchan Kim 2020-01-20 8:03 ` Michal Hocko 2020-01-20 8:03 ` Michal Hocko 2020-01-20 10:24 ` Kirill Tkhai 2020-01-20 10:24 ` Kirill Tkhai 2020-01-20 11:27 ` Michal Hocko 2020-01-20 11:27 ` Michal Hocko 2020-01-20 12:39 ` Kirill A. Shutemov 2020-01-20 13:24 ` Michal Hocko 2020-01-20 13:24 ` Michal Hocko 2020-01-20 14:21 ` Kirill A. Shutemov 2020-01-20 15:44 ` Michal Hocko 2020-01-20 15:44 ` Michal Hocko 2020-01-21 18:43 ` Minchan Kim 2020-01-21 18:43 ` Minchan Kim 2020-01-16 23:59 ` [PATCH v2 3/5] mm/madvise: employ mmget_still_valid for write lock Minchan Kim 2020-01-16 23:59 ` [PATCH v2 4/5] mm/madvise: allow KSM hints for remote API Minchan Kim 2020-01-17 10:13 ` Kirill Tkhai 2020-01-17 10:13 ` Kirill Tkhai 2020-01-17 12:34 ` Oleksandr Natalenko 2020-01-17 12:34 ` Oleksandr Natalenko 2020-01-21 17:45 ` Minchan Kim 2020-01-21 17:45 ` Minchan Kim 2020-01-16 23:59 ` [PATCH v2 5/5] mm: support both pid and pidfd for process_madvise Minchan Kim
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200122100233.GT29276@dhcp22.suse.cz \ --to=mhocko@kernel.org \ --cc=akpm@linux-foundation.org \ --cc=bgeffon@google.com \ --cc=christian.brauner@ubuntu.com \ --cc=dancol@google.com \ --cc=hannes@cmpxchg.org \ --cc=joaodias@google.com \ --cc=kirill@shutemov.name \ --cc=ktkhai@virtuozzo.com \ --cc=linux-api@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=minchan@kernel.org \ --cc=oleksandr@redhat.com \ --cc=shakeelb@google.com \ --cc=sjpark@amazon.com \ --cc=sjpark@amazon.de \ --cc=sonnyrao@google.com \ --cc=sspatil@google.com \ --cc=surenb@google.com \ --cc=timmurray@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.