All of lore.kernel.org
 help / color / mirror / Atom feed
From: John Stultz <john.stultz@linaro.org>
To: Dave Hansen <dave@sr71.net>, "H. Peter Anvin" <hpa@zytor.com>,
	Johannes Weiner <hannes@cmpxchg.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Android Kernel Team <kernel-team@android.com>,
	Robert Love <rlove@google.com>, Mel Gorman <mel@csn.ul.ie>,
	Hugh Dickins <hughd@google.com>, Rik van Riel <riel@redhat.com>,
	Dmitry Adamushko <dmitry.adamushko@gmail.com>,
	Neil Brown <neilb@suse.de>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Mike Hommey <mh@glandium.org>, Taras Glek <tglek@mozilla.com>,
	Jan Kara <jack@suse.cz>,
	KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
	Michel Lespinasse <walken@google.com>,
	Minchan Kim <minchan@kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder
Date: Tue, 01 Apr 2014 21:12:44 -0700	[thread overview]
Message-ID: <533B8E3C.3090606@linaro.org> (raw)
In-Reply-To: <533B4555.3000608@sr71.net>

On 04/01/2014 04:01 PM, Dave Hansen wrote:
> On 04/01/2014 02:35 PM, H. Peter Anvin wrote:
>> On 04/01/2014 02:21 PM, Johannes Weiner wrote:
>>> Either way, optimistic volatile pointers are nowhere near as
>>> transparent to the application as the above description suggests,
>>> which makes this usecase not very interesting, IMO.
>> ... however, I think you're still derating the value way too much.  The
>> case of user space doing elastic memory management is more and more
>> common, and for a lot of those applications it is perfectly reasonable
>> to either not do system calls or to have to devolatilize first.
> The SIGBUS is only in cases where the memory is set as volatile and
> _then_ accessed, right?
Not just set volatile and then accessed, but when a volatile page has
been purged and then accessed without being made non-volatile.


> John, this was something that the Mozilla guys asked for, right?  Any
> idea why this isn't ever a problem for them?
So one of their use cases for it is for library text. Basically they
want to decompress a compressed library file into memory. Then they plan
to mark the uncompressed pages volatile, and then be able to call into
it. Ideally for them, the kernel would only purge cold pages, leaving
the hot pages in memory. When they traverse a purged page, they handle
the SIGBUS and patch the page up.

Now.. this is not what I'd consider a normal use case, but was hoping to
illustrate some of the more interesting uses and demonstrate the
interfaces flexibility.

Also it provided a clear example of benefits to doing LRU based
cold-page purging rather then full object purging. Though I think the
same could be demonstrated in a simpler case of a large cache of objects
that the applications wants to mark volatile in one pass, unmarking
sub-objects as it needs.

thanks
-john


WARNING: multiple messages have this Message-ID (diff)
From: John Stultz <john.stultz@linaro.org>
To: Dave Hansen <dave@sr71.net>, "H. Peter Anvin" <hpa@zytor.com>,
	Johannes Weiner <hannes@cmpxchg.org>
Cc: LKML <linux-kernel@vger.kernel.org>,
	Andrew Morton <akpm@linux-foundation.org>,
	Android Kernel Team <kernel-team@android.com>,
	Robert Love <rlove@google.com>, Mel Gorman <mel@csn.ul.ie>,
	Hugh Dickins <hughd@google.com>, Rik van Riel <riel@redhat.com>,
	Dmitry Adamushko <dmitry.adamushko@gmail.com>,
	Neil Brown <neilb@suse.de>,
	Andrea Arcangeli <aarcange@redhat.com>,
	Mike Hommey <mh@glandium.org>, Taras Glek <tglek@mozilla.com>,
	Jan Kara <jack@suse.cz>,
	KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
	Michel Lespinasse <walken@google.com>,
	Minchan Kim <minchan@kernel.org>,
	"linux-mm@kvack.org" <linux-mm@kvack.org>
Subject: Re: [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder
Date: Tue, 01 Apr 2014 21:12:44 -0700	[thread overview]
Message-ID: <533B8E3C.3090606@linaro.org> (raw)
In-Reply-To: <533B4555.3000608@sr71.net>

On 04/01/2014 04:01 PM, Dave Hansen wrote:
> On 04/01/2014 02:35 PM, H. Peter Anvin wrote:
>> On 04/01/2014 02:21 PM, Johannes Weiner wrote:
>>> Either way, optimistic volatile pointers are nowhere near as
>>> transparent to the application as the above description suggests,
>>> which makes this usecase not very interesting, IMO.
>> ... however, I think you're still derating the value way too much.  The
>> case of user space doing elastic memory management is more and more
>> common, and for a lot of those applications it is perfectly reasonable
>> to either not do system calls or to have to devolatilize first.
> The SIGBUS is only in cases where the memory is set as volatile and
> _then_ accessed, right?
Not just set volatile and then accessed, but when a volatile page has
been purged and then accessed without being made non-volatile.


> John, this was something that the Mozilla guys asked for, right?  Any
> idea why this isn't ever a problem for them?
So one of their use cases for it is for library text. Basically they
want to decompress a compressed library file into memory. Then they plan
to mark the uncompressed pages volatile, and then be able to call into
it. Ideally for them, the kernel would only purge cold pages, leaving
the hot pages in memory. When they traverse a purged page, they handle
the SIGBUS and patch the page up.

Now.. this is not what I'd consider a normal use case, but was hoping to
illustrate some of the more interesting uses and demonstrate the
interfaces flexibility.

Also it provided a clear example of benefits to doing LRU based
cold-page purging rather then full object purging. Though I think the
same could be demonstrated in a simpler case of a large cache of objects
that the applications wants to mark volatile in one pass, unmarking
sub-objects as it needs.

thanks
-john

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2014-04-02  4:12 UTC|newest]

Thread overview: 112+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-03-21 21:17 [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder John Stultz
2014-03-21 21:17 ` John Stultz
2014-03-21 21:17 ` [PATCH 1/5] vrange: Add vrange syscall and handle splitting/merging and marking vmas John Stultz
2014-03-21 21:17   ` John Stultz
2014-03-23 12:20   ` Jan Kara
2014-03-23 12:20     ` Jan Kara
2014-03-23 20:34     ` John Stultz
2014-03-23 20:34       ` John Stultz
2014-03-23 16:50   ` KOSAKI Motohiro
2014-03-23 16:50     ` KOSAKI Motohiro
2014-04-08 18:52     ` John Stultz
2014-04-08 18:52       ` John Stultz
2014-03-21 21:17 ` [PATCH 2/5] vrange: Add purged page detection on setting memory non-volatile John Stultz
2014-03-21 21:17   ` John Stultz
2014-03-23 12:29   ` Jan Kara
2014-03-23 12:29     ` Jan Kara
2014-03-23 20:21     ` John Stultz
2014-03-23 20:21       ` John Stultz
2014-03-23 17:42   ` KOSAKI Motohiro
2014-03-23 17:42     ` KOSAKI Motohiro
2014-04-07 18:37     ` John Stultz
2014-04-07 18:37       ` John Stultz
2014-04-07 22:14       ` KOSAKI Motohiro
2014-04-07 22:14         ` KOSAKI Motohiro
2014-04-08  3:09         ` John Stultz
2014-04-08  3:09           ` John Stultz
2014-03-23 17:50   ` KOSAKI Motohiro
2014-03-23 17:50     ` KOSAKI Motohiro
2014-03-23 20:26     ` John Stultz
2014-03-23 20:26       ` John Stultz
2014-03-23 21:50       ` KOSAKI Motohiro
2014-03-23 21:50         ` KOSAKI Motohiro
2014-04-09 18:29         ` John Stultz
2014-04-09 18:29           ` John Stultz
2014-03-21 21:17 ` [PATCH 3/5] vrange: Add page purging logic & SIGBUS trap John Stultz
2014-03-21 21:17   ` John Stultz
2014-03-23 23:44   ` KOSAKI Motohiro
2014-03-23 23:44     ` KOSAKI Motohiro
2014-04-10 18:49     ` John Stultz
2014-04-10 18:49       ` John Stultz
2014-03-21 21:17 ` [PATCH 4/5] vrange: Set affected pages referenced when marking volatile John Stultz
2014-03-21 21:17   ` John Stultz
2014-03-24  0:01   ` KOSAKI Motohiro
2014-03-24  0:01     ` KOSAKI Motohiro
2014-03-21 21:17 ` [PATCH 5/5] vmscan: Age anonymous memory even when swap is off John Stultz
2014-03-21 21:17   ` John Stultz
2014-03-24 17:33   ` Rik van Riel
2014-03-24 17:33     ` Rik van Riel
2014-03-24 18:04     ` John Stultz
2014-03-24 18:04       ` John Stultz
2014-04-01 21:21 ` [PATCH 0/5] Volatile Ranges (v12) & LSF-MM discussion fodder Johannes Weiner
2014-04-01 21:21   ` Johannes Weiner
2014-04-01 21:34   ` H. Peter Anvin
2014-04-01 21:34     ` H. Peter Anvin
2014-04-01 21:35   ` H. Peter Anvin
2014-04-01 21:35     ` H. Peter Anvin
2014-04-01 23:01     ` Dave Hansen
2014-04-01 23:01       ` Dave Hansen
2014-04-02  4:12       ` John Stultz [this message]
2014-04-02  4:12         ` John Stultz
2014-04-02 16:36         ` Johannes Weiner
2014-04-02 16:36           ` Johannes Weiner
2014-04-02 17:40           ` John Stultz
2014-04-02 17:40             ` John Stultz
2014-04-02 17:58             ` Johannes Weiner
2014-04-02 17:58               ` Johannes Weiner
2014-04-02 19:01               ` John Stultz
2014-04-02 19:01                 ` John Stultz
2014-04-02 19:47                 ` Johannes Weiner
2014-04-02 19:47                   ` Johannes Weiner
2014-04-02 20:13                   ` John Stultz
2014-04-02 20:13                     ` John Stultz
2014-04-02 22:44                     ` Jan Kara
2014-04-02 22:44                       ` Jan Kara
2014-04-11 19:32                     ` John Stultz
2014-04-11 19:32                       ` John Stultz
2014-04-07  5:48             ` Minchan Kim
2014-04-07  5:48               ` Minchan Kim
2014-04-08  4:32             ` Kevin Easton
2014-04-08  3:38               ` John Stultz
2014-04-08  3:38                 ` John Stultz
2014-04-07  5:24           ` Minchan Kim
2014-04-07  5:24             ` Minchan Kim
2014-04-02  4:03   ` John Stultz
2014-04-02  4:03     ` John Stultz
2014-04-02  4:07     ` H. Peter Anvin
2014-04-02  4:07       ` H. Peter Anvin
2014-04-02 16:30     ` Johannes Weiner
2014-04-02 16:30       ` Johannes Weiner
2014-04-02 16:32       ` H. Peter Anvin
2014-04-02 16:32         ` H. Peter Anvin
2014-04-02 16:37         ` H. Peter Anvin
2014-04-02 17:18           ` Johannes Weiner
2014-04-02 17:18             ` Johannes Weiner
2014-04-02 17:40             ` Dave Hansen
2014-04-02 17:40               ` Dave Hansen
2014-04-02 17:48               ` John Stultz
2014-04-02 17:48                 ` John Stultz
2014-04-02 18:07                 ` Johannes Weiner
2014-04-02 18:07                   ` Johannes Weiner
2014-04-02 19:37                   ` John Stultz
2014-04-02 19:37                     ` John Stultz
2014-04-02 18:31     ` Andrea Arcangeli
2014-04-02 18:31       ` Andrea Arcangeli
2014-04-02 19:27       ` Johannes Weiner
2014-04-02 19:27         ` Johannes Weiner
2014-04-07  6:19         ` Minchan Kim
2014-04-07  6:19           ` Minchan Kim
2014-04-02 19:51       ` John Stultz
2014-04-02 19:51         ` John Stultz
2014-04-07  6:11       ` Minchan Kim
2014-04-07  6:11         ` Minchan Kim

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=533B8E3C.3090606@linaro.org \
    --to=john.stultz@linaro.org \
    --cc=aarcange@redhat.com \
    --cc=akpm@linux-foundation.org \
    --cc=dave@sr71.net \
    --cc=dmitry.adamushko@gmail.com \
    --cc=hannes@cmpxchg.org \
    --cc=hpa@zytor.com \
    --cc=hughd@google.com \
    --cc=jack@suse.cz \
    --cc=kernel-team@android.com \
    --cc=kosaki.motohiro@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mel@csn.ul.ie \
    --cc=mh@glandium.org \
    --cc=minchan@kernel.org \
    --cc=neilb@suse.de \
    --cc=riel@redhat.com \
    --cc=rlove@google.com \
    --cc=tglek@mozilla.com \
    --cc=walken@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.