From: Minchan Kim <minchan@kernel.org>
To: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Cc: linux-mm <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
leonid.moiseichuk@nokia.com, penberg@kernel.org,
Rik van Riel <riel@redhat.com>,
mel@csn.ul.ie, rientjes@google.com,
KOSAKI Motohiro <kosaki.motohiro@gmail.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Marcelo Tosatti <mtosatti@redhat.com>,
Andrew Morton <akpm@linux-foundation.org>,
Ronen Hod <rhod@redhat.com>
Subject: Re: [RFC 2/3] vmscan hook
Date: Wed, 18 Jan 2012 08:08:01 +0900 [thread overview]
Message-ID: <20120117230801.GA903@barrios-desktop.redhat.com> (raw)
In-Reply-To: <20120117190512.047d3a03.kamezawa.hiroyu@jp.fujitsu.com>
On Tue, Jan 17, 2012 at 07:05:12PM +0900, KAMEZAWA Hiroyuki wrote:
> On Tue, 17 Jan 2012 18:13:56 +0900
> Minchan Kim <minchan@kernel.org> wrote:
>
> > On Tue, Jan 17, 2012 at 05:39:32PM +0900, KAMEZAWA Hiroyuki wrote:
> > > On Tue, 17 Jan 2012 17:13:57 +0900
> > > Minchan Kim <minchan@kernel.org> wrote:
> > >
> > >
> > > > + /*
> > > > + * We want to avoid dropping page cache excessively
> > > > + * in no swap system
> > > > + */
> > > > + if (nr_swap_pages <= 0) {
> > > > + free = zone_page_state(mz->zone, NR_FREE_PAGES);
> > > > + file = zone_page_state(mz->zone, NR_ACTIVE_FILE) +
> > > > + zone_page_state(mz->zone, NR_INACTIVE_FILE);
> > > > + /*
> > > > + * If we have very few page cache pages,
> > > > + * notify to user
> > > > + */
> > > > + if (file < free)
> > > > + low_mem = true;
> > > > + }
> > >
> > > I can't understand why you think you can check lowmem condition by "file < free".
> >
> > The reason I thought so is I want to maintain some page cache to some degree.
> > But I admit It's very naive heuristic and should be improved.
> >
> > > And I don't think using per-zone data is good.
> > > (I'm not sure how many zones embeded guys using..)
> >
> > Agree. In case of swapless system, we need another heuristic.
> >
> > >
> > > Another idea:
> > > 1. can't we use some technique like cleancache to detect the condition ?
> >
> > I totally forgot cleancache approach. Could you remind that?
> >
>
> Similar to 'victim cache'. Then, cache some clean pages somewhere when
> vmscan pageout it.
>
> page -> vmscan's pageout -> cleancache -> may be discarded.
>
> If a filesystem look up a page which is in a cleancache, cache-hit and
> bring it back to radix-tree. If not, read from disk again.
> And cleancache for swap(frontswap) was posted, too.
I am not sure this can prevent swapout.
I think it ends up evicting pages into swap devices.
>
>
> > > 2. can't we measure page-in/page-out distance by recording something ?
> >
> > I can't understand your point. What's relation does it with swapout prevent?
> >
>
> If distance between pageout -> pagein is short, it means thrashing.
> For example, recoding the timestamp when the page(mapping, index) was
> paged-out, and check it at page-in.
Our goal is prevent swapout. When we found thrashing, it's too late.
>
>
> > > 3. NR_ANON + NR_FILE_MAPPED can't mean the amount of core memory if we can
> > > ignore the data file cache ?
> >
> > It's good but how do we define some amount?
> > It's very vague but I guess we can get a good idea from that.
> > Perhaps, you already has it.
> >
>
> Hm, a rough idea is...
>
> - we now have rss counter per mm.
> - mapped anon
> - mapped file
> - swapents
>
> Ok, here, add one more counter.
>
> - paged-out file. (I think this can be recorded in pte.)
> +1 when try_to_unmap_file() unmaps it.
> -1 when a page is back or unmapped.
>
> Then, scanning all tasks. Then,
>
> mapped_anon + mapped_file
> active_map_ratio = ----------------------------------------------------- * 100
> mapped_anon + mapped_file + swapents + paged_out_file
>
> Ok, how to use this value...
>
> Like memcg's threshold notify interface, you can change the mem_notify interface
> to use eventfd() as
>
> <event_fd, fd of /dev/mem_notify, threshold of active_map_ratio>
>
> This will inform you an event when active_map_ratio crosses passed threshold.
>
> complicated ?
Yes. :)
I want to make simple if possible.
>
>
> > > 4. how about checking kswapd's busy status ?
> >
> > Could you elaborate on your idea?
> >
>
> I just thought kswapd may not stop when the situation is very bad.
As I said eariler, the goal is prevent swap.
When we found kswapd is busy, it might many pages are already swapped-out so it's too late.
>
> Thanks,
> -Kame
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2012-01-17 23:08 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-01-17 8:13 [RFC 0/3] low memory notify Minchan Kim
2012-01-17 8:13 ` [RFC 1/3] /dev/low_mem_notify Minchan Kim
2012-01-17 9:27 ` Pekka Enberg
2012-01-17 16:35 ` Rik van Riel
2012-01-17 18:51 ` Pekka Enberg
2012-01-17 19:30 ` Rik van Riel
2012-01-17 19:49 ` Pekka Enberg
2012-01-17 19:54 ` Pekka Enberg
2012-01-17 19:57 ` Pekka Enberg
2012-01-17 23:20 ` Minchan Kim
2012-01-18 7:16 ` Pekka Enberg
2012-01-18 7:49 ` Minchan Kim
2012-01-18 9:06 ` leonid.moiseichuk
2012-01-18 9:15 ` Pekka Enberg
2012-01-18 9:41 ` leonid.moiseichuk
2012-01-18 10:40 ` Pekka Enberg
2012-01-18 10:44 ` leonid.moiseichuk
2012-01-18 23:34 ` Ronen Hod
2012-01-19 7:25 ` Pekka Enberg
2012-01-19 9:05 ` Ronen Hod
2012-01-19 9:10 ` Pekka Enberg
2012-01-19 9:20 ` Ronen Hod
2012-01-19 10:53 ` leonid.moiseichuk
2012-01-19 11:07 ` Pekka Enberg
2012-01-19 11:54 ` leonid.moiseichuk
2012-01-19 11:59 ` Pekka Enberg
2012-01-19 12:06 ` Pekka Enberg
2012-01-24 15:38 ` Marcelo Tosatti
2012-01-24 16:08 ` Ronen Hod
2012-01-24 18:10 ` Marcelo Tosatti
2012-01-25 8:52 ` Ronen Hod
2012-01-25 10:12 ` Marcelo Tosatti
2012-01-25 10:48 ` Ronen Hod
2012-01-26 16:17 ` Marcelo Tosatti
2012-01-24 16:10 ` Pekka Enberg
2012-01-24 18:29 ` Marcelo Tosatti
2012-01-25 8:19 ` leonid.moiseichuk
2012-01-19 7:34 ` Pekka Enberg
2012-01-24 16:22 ` Arnd Bergmann
2012-01-18 14:30 ` Rik van Riel
2012-01-18 15:29 ` Pekka Enberg
2012-01-24 15:40 ` Marcelo Tosatti
2012-01-24 16:01 ` Pekka Enberg
2012-01-24 16:25 ` Arnd Bergmann
2012-01-24 18:32 ` Marcelo Tosatti
2012-01-24 21:57 ` Jonathan Corbet
2012-01-17 9:45 ` Pekka Enberg
2012-01-17 8:13 ` [RFC 2/3] vmscan hook Minchan Kim
2012-01-17 8:39 ` KAMEZAWA Hiroyuki
2012-01-17 9:13 ` Minchan Kim
2012-01-17 10:05 ` KAMEZAWA Hiroyuki
2012-01-17 23:08 ` Minchan Kim [this message]
2012-01-18 0:18 ` KAMEZAWA Hiroyuki
2012-01-18 14:17 ` Rik van Riel
2012-01-19 2:25 ` KAMEZAWA Hiroyuki
2012-01-19 14:42 ` Rik van Riel
2012-01-20 0:24 ` KAMEZAWA Hiroyuki
2012-01-17 8:13 ` [RFC 3/3] test program Minchan Kim
2012-01-17 14:38 ` [RFC 0/3] low memory notify Colin Walters
2012-01-17 15:04 ` Pekka Enberg
2012-01-17 16:44 ` Rik van Riel
2012-01-17 17:16 ` Olof Johansson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120117230801.GA903@barrios-desktop.redhat.com \
--to=minchan@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=hannes@cmpxchg.org \
--cc=kamezawa.hiroyu@jp.fujitsu.com \
--cc=kosaki.motohiro@gmail.com \
--cc=leonid.moiseichuk@nokia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mel@csn.ul.ie \
--cc=mtosatti@redhat.com \
--cc=penberg@kernel.org \
--cc=rhod@redhat.com \
--cc=riel@redhat.com \
--cc=rientjes@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).