From: Shakeel Butt <firstname.lastname@example.org> To: Anshuman Khandual <email@example.com> Cc: Tim Murray <firstname.lastname@example.org>, Minchan Kim <email@example.com>, Andrew Morton <firstname.lastname@example.org>, LKML <email@example.com>, linux-mm <firstname.lastname@example.org>, Michal Hocko <email@example.com>, Johannes Weiner <firstname.lastname@example.org>, Joel Fernandes <email@example.com>, Suren Baghdasaryan <firstname.lastname@example.org>, Daniel Colascione <email@example.com>, Sonny Rao <firstname.lastname@example.org>, Brian Geffon <email@example.com>, firstname.lastname@example.org Subject: Re: [RFC 0/7] introduce memory hinting API for external process Date: Tue, 21 May 2019 05:56:59 -0700 Message-ID: <CALvZod6ioRxSi7tHB-uSTxN1-hsxD+8O3mfFAjaqdsimjUVmcw@mail.gmail.com> (raw) In-Reply-To: <email@example.com> On Mon, May 20, 2019 at 7:55 PM Anshuman Khandual <firstname.lastname@example.org> wrote: > > > > On 05/20/2019 10:29 PM, Tim Murray wrote: > > On Sun, May 19, 2019 at 11:37 PM Anshuman Khandual > > <email@example.com> wrote: > >> > >> Or Is the objective here is reduce the number of processes which get killed by > >> lmkd by triggering swapping for the unused memory (user hinted) sooner so that > >> they dont get picked by lmkd. Under utilization for zram hardware is a concern > >> here as well ? > > > > The objective is to avoid some instances of memory pressure by > > proactively swapping pages that userspace knows to be cold before > > those pages reach the end of the LRUs, which in turn can prevent some > > apps from being killed by lmk/lmkd. As soon as Android userspace knows > > that an application is not being used and is only resident to improve > > performance if the user returns to that app, we can kick off > > process_madvise on that process's pages (or some portion of those > > pages) in a power-efficient way to reduce memory pressure long before > > the system hits the free page watermark. This allows the system more > > time to put pages into zram versus waiting for the watermark to > > trigger kswapd, which decreases the likelihood that later memory > > allocations will cause enough pressure to trigger a kill of one of > > these apps. > > So this opens up bit of LRU management to user space hints. Also because the app > in itself wont know about the memory situation of the entire system, new system > call needs to be called from an external process. > > > > >> Swapping out memory into zram wont increase the latency for a hot start ? Or > >> is it because as it will prevent a fresh cold start which anyway will be slower > >> than a slow hot start. Just being curious. > > > > First, not all swapped pages will be reloaded immediately once an app > > is resumed. We've found that an app's working set post-process_madvise > > is significantly smaller than what an app allocates when it first > > launches (see the delta between pswpin and pswpout in Minchan's > > results). Presumably because of this, faulting to fetch from zram does > > pswpin 417613 1392647 975034 233.00 > pswpout 1274224 2661731 1387507 108.00 > > IIUC the swap-in ratio is way higher in comparison to that of swap out. Is that > always the case ? Or it tend to swap out from an active area of the working set > which faulted back again. > > > not seem to introduce a noticeable hot start penalty, not does it > > cause an increase in performance problems later in the app's > > lifecycle. I've measured with and without process_madvise, and the > > differences are within our noise bounds. Second, because we're not > > That is assuming that post process_madvise() working set for the application is > always smaller. There is another challenge. The external process should ideally > have the knowledge of active areas of the working set for an application in > question for it to invoke process_madvise() correctly to prevent such scenarios. > > > preemptively evicting file pages and only making them more likely to > > be evicted when there's already memory pressure, we avoid the case > > where we process_madvise an app then immediately return to the app and > > reload all file pages in the working set even though there was no > > intervening memory pressure. Our initial version of this work evicted > > That would be the worst case scenario which should be avoided. Memory pressure > must be a parameter before actually doing the swap out. But pages if know to be > inactive/cold can be marked high priority to be swapped out. > > > file pages preemptively and did cause a noticeable slowdown (~15%) for > > that case; this patch set avoids that slowdown. Finally, the benefit > > from avoiding cold starts is huge. The performance improvement from > > having a hot start instead of a cold start ranges from 3x for very > > small apps to 50x+ for larger apps like high-fidelity games. > > Is there any other real world scenario apart from this app based ecosystem where > user hinted LRU management might be helpful ? Just being curious. Thanks for the > detailed explanation. I will continue looking into this series. Chrome OS is another real world use-case for this user hinted LRU management approach by proactively reclaiming reclaim from tabs not accessed by the user for some time.
next prev parent reply index Thread overview: 68+ messages / expand[flat|nested] mbox.gz Atom feed top [not found] <firstname.lastname@example.org> [not found] ` <email@example.com> 2019-05-20 8:16 ` [RFC 1/7] mm: introduce MADV_COOL Michal Hocko 2019-05-20 8:19 ` Michal Hocko 2019-05-20 15:08 ` Suren Baghdasaryan 2019-05-20 22:55 ` Minchan Kim 2019-05-20 22:54 ` Minchan Kim 2019-05-21 6:04 ` Michal Hocko 2019-05-21 9:11 ` Minchan Kim 2019-05-21 10:05 ` Michal Hocko [not found] ` <firstname.lastname@example.org> 2019-05-20 8:27 ` [RFC 3/7] mm: introduce MADV_COLD Michal Hocko 2019-05-20 23:00 ` Minchan Kim 2019-05-21 6:08 ` Michal Hocko 2019-05-21 9:13 ` Minchan Kim [not found] ` <email@example.com> 2019-05-20 9:18 ` [RFC 5/7] mm: introduce external memory hinting API Michal Hocko 2019-05-21 2:41 ` Minchan Kim 2019-05-21 6:17 ` Michal Hocko 2019-05-21 10:32 ` Minchan Kim [not found] ` <firstname.lastname@example.org> 2019-05-20 9:22 ` [RFC 6/7] mm: extend process_madvise syscall to support vector arrary Michal Hocko 2019-05-21 2:48 ` Minchan Kim 2019-05-21 6:24 ` Michal Hocko 2019-05-21 10:26 ` Minchan Kim 2019-05-21 10:37 ` Michal Hocko 2019-05-27 7:49 ` Minchan Kim 2019-05-29 10:08 ` Daniel Colascione 2019-05-29 10:33 ` Michal Hocko 2019-05-30 2:17 ` Minchan Kim 2019-05-30 6:57 ` Michal Hocko 2019-05-30 8:02 ` Minchan Kim 2019-05-30 16:19 ` Daniel Colascione 2019-05-30 18:47 ` Michal Hocko [not found] ` <email@example.com> 2019-05-20 9:28 ` [RFC 7/7] mm: madvise support MADV_ANONYMOUS_FILTER and MADV_FILE_FILTER Michal Hocko 2019-05-21 2:55 ` Minchan Kim 2019-05-21 6:26 ` Michal Hocko 2019-05-27 7:58 ` Minchan Kim 2019-05-27 12:44 ` Michal Hocko 2019-05-28 3:26 ` Minchan Kim 2019-05-28 6:29 ` Michal Hocko 2019-05-28 8:13 ` Minchan Kim 2019-05-28 8:31 ` Daniel Colascione 2019-05-28 8:49 ` Minchan Kim 2019-05-28 9:08 ` Michal Hocko 2019-05-28 9:39 ` Daniel Colascione 2019-05-28 10:33 ` Michal Hocko 2019-05-28 11:21 ` Daniel Colascione 2019-05-28 11:49 ` Michal Hocko 2019-05-28 12:11 ` Daniel Colascione 2019-05-28 12:32 ` Michal Hocko 2019-05-28 10:32 ` Minchan Kim 2019-05-28 10:41 ` Michal Hocko 2019-05-28 11:12 ` Minchan Kim 2019-05-28 11:28 ` Michal Hocko 2019-05-28 11:42 ` Daniel Colascione 2019-05-28 11:56 ` Michal Hocko 2019-05-28 12:18 ` Daniel Colascione 2019-05-28 12:38 ` Michal Hocko 2019-05-28 12:10 ` Minchan Kim 2019-05-28 11:44 ` Minchan Kim 2019-05-28 11:51 ` Daniel Colascione 2019-05-28 12:06 ` Michal Hocko 2019-05-28 12:22 ` Minchan Kim 2019-05-28 11:28 ` Daniel Colascione 2019-05-21 15:33 ` Johannes Weiner 2019-05-22 1:50 ` Minchan Kim 2019-05-20 9:28 ` [RFC 0/7] introduce memory hinting API for external process Michal Hocko [not found] ` <20190520164605.GA11665@cmpxchg.org> [not found] ` <20190521043950.GJ10039@google.com> 2019-05-21 6:32 ` Michal Hocko [not found] ` <20190521014452.GA6738@bombadil.infradead.org> 2019-05-21 6:34 ` Michal Hocko 2019-05-21 12:53 ` Shakeel Butt [not found] ` <firstname.lastname@example.org> [not found] ` <CAEe=Sxka3Q3vX+7aWUJGKicM+a9Px0rrusyL+5bB1w4ywF6N4Q@mail.gmail.com> [not found] ` <email@example.com> 2019-05-21 12:56 ` Shakeel Butt [this message] 2019-05-22 4:23 ` Brian Geffon
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=CALvZod6ioRxSi7tHB-uSTxN1-hsxD+8O3mfFAjaqdsimjUVmcw@mail.gmail.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
Linux-api Archive on lore.kernel.org Archives are clonable: git clone --mirror https://lore.kernel.org/linux-api/0 linux-api/git/0.git # If you have public-inbox 1.1+ installed, you may # initialize and index your mirror using the following commands: public-inbox-init -V2 linux-api linux-api/ https://lore.kernel.org/linux-api \ email@example.com public-inbox-index linux-api Example config snippet for mirrors Newsgroup available over NNTP: nntp://nntp.lore.kernel.org/org.kernel.vger.linux-api AGPL code for this site: git clone https://public-inbox.org/public-inbox.git