From: John Stultz <john.stultz@linaro.org> To: Dave Hansen <dave@sr71.net>, "H. Peter Anvin" <hpa@zytor.com>, Johannes Weiner <hannes@cmpxchg.org> Cc: LKML <linux-kernel@vger.kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Android Kernel Team <kernel-team@android.com>, Robert Love <rlove@google.com>, Mel Gorman <mel@csn.ul.ie>, Hugh Dickins <hughd@google.com>, Rik van Riel <riel@redhat.com>, Dmitry Adamushko <dmitry.adamushko@gmail.com>, Neil Brown <neilb@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Mike Hommey <mh@glandium.org>, Taras Glek <tglek@mozilla.com>, Jan Kara <jack@suse.cz>, KOSAKI Motohiro <kosaki.motohiro@gmail.com>, Michel Lespinasse <walken@google.com>, Minchan Kim <minchan@kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org> Subject: Re: [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder Date: Tue, 01 Apr 2014 21:12:44 -0700 [thread overview] Message-ID: <533B8E3C.3090606@linaro.org> (raw) In-Reply-To: <533B4555.3000608@sr71.net> On 04/01/2014 04:01 PM, Dave Hansen wrote: > On 04/01/2014 02:35 PM, H. Peter Anvin wrote: >> On 04/01/2014 02:21 PM, Johannes Weiner wrote: >>> Either way, optimistic volatile pointers are nowhere near as >>> transparent to the application as the above description suggests, >>> which makes this usecase not very interesting, IMO. >> ... however, I think you're still derating the value way too much. The >> case of user space doing elastic memory management is more and more >> common, and for a lot of those applications it is perfectly reasonable >> to either not do system calls or to have to devolatilize first. > The SIGBUS is only in cases where the memory is set as volatile and > _then_ accessed, right? Not just set volatile and then accessed, but when a volatile page has been purged and then accessed without being made non-volatile. > John, this was something that the Mozilla guys asked for, right? Any > idea why this isn't ever a problem for them? So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up. Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility. Also it provided a clear example of benefits to doing LRU based cold-page purging rather then full object purging. Though I think the same could be demonstrated in a simpler case of a large cache of objects that the applications wants to mark volatile in one pass, unmarking sub-objects as it needs. thanks -john
WARNING: multiple messages have this Message-ID (diff)
From: John Stultz <john.stultz@linaro.org> To: Dave Hansen <dave@sr71.net>, "H. Peter Anvin" <hpa@zytor.com>, Johannes Weiner <hannes@cmpxchg.org> Cc: LKML <linux-kernel@vger.kernel.org>, Andrew Morton <akpm@linux-foundation.org>, Android Kernel Team <kernel-team@android.com>, Robert Love <rlove@google.com>, Mel Gorman <mel@csn.ul.ie>, Hugh Dickins <hughd@google.com>, Rik van Riel <riel@redhat.com>, Dmitry Adamushko <dmitry.adamushko@gmail.com>, Neil Brown <neilb@suse.de>, Andrea Arcangeli <aarcange@redhat.com>, Mike Hommey <mh@glandium.org>, Taras Glek <tglek@mozilla.com>, Jan Kara <jack@suse.cz>, KOSAKI Motohiro <kosaki.motohiro@gmail.com>, Michel Lespinasse <walken@google.com>, Minchan Kim <minchan@kernel.org>, "linux-mm@kvack.org" <linux-mm@kvack.org> Subject: Re: [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder Date: Tue, 01 Apr 2014 21:12:44 -0700 [thread overview] Message-ID: <533B8E3C.3090606@linaro.org> (raw) In-Reply-To: <533B4555.3000608@sr71.net> On 04/01/2014 04:01 PM, Dave Hansen wrote: > On 04/01/2014 02:35 PM, H. Peter Anvin wrote: >> On 04/01/2014 02:21 PM, Johannes Weiner wrote: >>> Either way, optimistic volatile pointers are nowhere near as >>> transparent to the application as the above description suggests, >>> which makes this usecase not very interesting, IMO. >> ... however, I think you're still derating the value way too much. The >> case of user space doing elastic memory management is more and more >> common, and for a lot of those applications it is perfectly reasonable >> to either not do system calls or to have to devolatilize first. > The SIGBUS is only in cases where the memory is set as volatile and > _then_ accessed, right? Not just set volatile and then accessed, but when a volatile page has been purged and then accessed without being made non-volatile. > John, this was something that the Mozilla guys asked for, right? Any > idea why this isn't ever a problem for them? So one of their use cases for it is for library text. Basically they want to decompress a compressed library file into memory. Then they plan to mark the uncompressed pages volatile, and then be able to call into it. Ideally for them, the kernel would only purge cold pages, leaving the hot pages in memory. When they traverse a purged page, they handle the SIGBUS and patch the page up. Now.. this is not what I'd consider a normal use case, but was hoping to illustrate some of the more interesting uses and demonstrate the interfaces flexibility. Also it provided a clear example of benefits to doing LRU based cold-page purging rather then full object purging. Though I think the same could be demonstrated in a simpler case of a large cache of objects that the applications wants to mark volatile in one pass, unmarking sub-objects as it needs. thanks -john -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2014-04-02 4:12 UTC|newest] Thread overview: 112+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-03-21 21:17 [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder John Stultz 2014-03-21 21:17 ` John Stultz 2014-03-21 21:17 ` [PATCH 1/5] vrange: Add vrange syscall and handle splitting/merging and marking vmas John Stultz 2014-03-21 21:17 ` John Stultz 2014-03-23 12:20 ` Jan Kara 2014-03-23 12:20 ` Jan Kara 2014-03-23 20:34 ` John Stultz 2014-03-23 20:34 ` John Stultz 2014-03-23 16:50 ` KOSAKI Motohiro 2014-03-23 16:50 ` KOSAKI Motohiro 2014-04-08 18:52 ` John Stultz 2014-04-08 18:52 ` John Stultz 2014-03-21 21:17 ` [PATCH 2/5] vrange: Add purged page detection on setting memory non-volatile John Stultz 2014-03-21 21:17 ` John Stultz 2014-03-23 12:29 ` Jan Kara 2014-03-23 12:29 ` Jan Kara 2014-03-23 20:21 ` John Stultz 2014-03-23 20:21 ` John Stultz 2014-03-23 17:42 ` KOSAKI Motohiro 2014-03-23 17:42 ` KOSAKI Motohiro 2014-04-07 18:37 ` John Stultz 2014-04-07 18:37 ` John Stultz 2014-04-07 22:14 ` KOSAKI Motohiro 2014-04-07 22:14 ` KOSAKI Motohiro 2014-04-08 3:09 ` John Stultz 2014-04-08 3:09 ` John Stultz 2014-03-23 17:50 ` KOSAKI Motohiro 2014-03-23 17:50 ` KOSAKI Motohiro 2014-03-23 20:26 ` John Stultz 2014-03-23 20:26 ` John Stultz 2014-03-23 21:50 ` KOSAKI Motohiro 2014-03-23 21:50 ` KOSAKI Motohiro 2014-04-09 18:29 ` John Stultz 2014-04-09 18:29 ` John Stultz 2014-03-21 21:17 ` [PATCH 3/5] vrange: Add page purging logic & SIGBUS trap John Stultz 2014-03-21 21:17 ` John Stultz 2014-03-23 23:44 ` KOSAKI Motohiro 2014-03-23 23:44 ` KOSAKI Motohiro 2014-04-10 18:49 ` John Stultz 2014-04-10 18:49 ` John Stultz 2014-03-21 21:17 ` [PATCH 4/5] vrange: Set affected pages referenced when marking volatile John Stultz 2014-03-21 21:17 ` John Stultz 2014-03-24 0:01 ` KOSAKI Motohiro 2014-03-24 0:01 ` KOSAKI Motohiro 2014-03-21 21:17 ` [PATCH 5/5] vmscan: Age anonymous memory even when swap is off John Stultz 2014-03-21 21:17 ` John Stultz 2014-03-24 17:33 ` Rik van Riel 2014-03-24 17:33 ` Rik van Riel 2014-03-24 18:04 ` John Stultz 2014-03-24 18:04 ` John Stultz 2014-04-01 21:21 ` [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder Johannes Weiner 2014-04-01 21:21 ` Johannes Weiner 2014-04-01 21:34 ` H. Peter Anvin 2014-04-01 21:34 ` H. Peter Anvin 2014-04-01 21:35 ` H. Peter Anvin 2014-04-01 21:35 ` H. Peter Anvin 2014-04-01 23:01 ` Dave Hansen 2014-04-01 23:01 ` Dave Hansen 2014-04-02 4:12 ` John Stultz [this message] 2014-04-02 4:12 ` John Stultz 2014-04-02 16:36 ` Johannes Weiner 2014-04-02 16:36 ` Johannes Weiner 2014-04-02 17:40 ` John Stultz 2014-04-02 17:40 ` John Stultz 2014-04-02 17:58 ` Johannes Weiner 2014-04-02 17:58 ` Johannes Weiner 2014-04-02 19:01 ` John Stultz 2014-04-02 19:01 ` John Stultz 2014-04-02 19:47 ` Johannes Weiner 2014-04-02 19:47 ` Johannes Weiner 2014-04-02 20:13 ` John Stultz 2014-04-02 20:13 ` John Stultz 2014-04-02 22:44 ` Jan Kara 2014-04-02 22:44 ` Jan Kara 2014-04-11 19:32 ` John Stultz 2014-04-11 19:32 ` John Stultz 2014-04-07 5:48 ` Minchan Kim 2014-04-07 5:48 ` Minchan Kim 2014-04-08 4:32 ` Kevin Easton 2014-04-08 3:38 ` John Stultz 2014-04-08 3:38 ` John Stultz 2014-04-07 5:24 ` Minchan Kim 2014-04-07 5:24 ` Minchan Kim 2014-04-02 4:03 ` John Stultz 2014-04-02 4:03 ` John Stultz 2014-04-02 4:07 ` H. Peter Anvin 2014-04-02 4:07 ` H. Peter Anvin 2014-04-02 16:30 ` Johannes Weiner 2014-04-02 16:30 ` Johannes Weiner 2014-04-02 16:32 ` H. Peter Anvin 2014-04-02 16:32 ` H. Peter Anvin 2014-04-02 16:37 ` H. Peter Anvin 2014-04-02 17:18 ` Johannes Weiner 2014-04-02 17:18 ` Johannes Weiner 2014-04-02 17:40 ` Dave Hansen 2014-04-02 17:40 ` Dave Hansen 2014-04-02 17:48 ` John Stultz 2014-04-02 17:48 ` John Stultz 2014-04-02 18:07 ` Johannes Weiner 2014-04-02 18:07 ` Johannes Weiner 2014-04-02 19:37 ` John Stultz 2014-04-02 19:37 ` John Stultz 2014-04-02 18:31 ` Andrea Arcangeli 2014-04-02 18:31 ` Andrea Arcangeli 2014-04-02 19:27 ` Johannes Weiner 2014-04-02 19:27 ` Johannes Weiner 2014-04-07 6:19 ` Minchan Kim 2014-04-07 6:19 ` Minchan Kim 2014-04-02 19:51 ` John Stultz 2014-04-02 19:51 ` John Stultz 2014-04-07 6:11 ` Minchan Kim 2014-04-07 6:11 ` Minchan Kim
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=533B8E3C.3090606@linaro.org \ --to=john.stultz@linaro.org \ --cc=aarcange@redhat.com \ --cc=akpm@linux-foundation.org \ --cc=dave@sr71.net \ --cc=dmitry.adamushko@gmail.com \ --cc=hannes@cmpxchg.org \ --cc=hpa@zytor.com \ --cc=hughd@google.com \ --cc=jack@suse.cz \ --cc=kernel-team@android.com \ --cc=kosaki.motohiro@gmail.com \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=mel@csn.ul.ie \ --cc=mh@glandium.org \ --cc=minchan@kernel.org \ --cc=neilb@suse.de \ --cc=riel@redhat.com \ --cc=rlove@google.com \ --cc=tglek@mozilla.com \ --cc=walken@google.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.