From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> To: Johannes Weiner <hannes@cmpxchg.org> Cc: "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "nishimura@mxp.nes.nec.co.jp" <nishimura@mxp.nes.nec.co.jp>, "balbir@linux.vnet.ibm.com" <balbir@linux.vnet.ibm.com> Subject: Re: [BUGFIX][PATCH 2/4] memcg: fix charge path for THP and allow early retirement Date: Fri, 28 Jan 2011 18:16:53 +0900 [thread overview] Message-ID: <20110128181653.240c73a0.kamezawa.hiroyu@jp.fujitsu.com> (raw) In-Reply-To: <20110128090242.GF2213@cmpxchg.org> On Fri, 28 Jan 2011 10:02:42 +0100 Johannes Weiner <hannes@cmpxchg.org> wrote: > On Fri, Jan 28, 2011 at 05:14:47PM +0900, KAMEZAWA Hiroyuki wrote: > > On Fri, 28 Jan 2011 08:57:24 +0100 > > Johannes Weiner <hannes@cmpxchg.org> wrote: > > > > > On Fri, Jan 28, 2011 at 12:26:08PM +0900, KAMEZAWA Hiroyuki wrote: > > > > From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > > > > > > > When THP is used, Hugepage size charge can happen. It's not handled > > > > correctly in mem_cgroup_do_charge(). For example, THP can fallback > > > > to small page allocation when HUGEPAGE allocation seems difficult > > > > or busy, but memory cgroup doesn't understand it and continue to > > > > try HUGEPAGE charging. And the worst thing is memory cgroup > > > > believes 'memory reclaim succeeded' if limit - usage > PAGE_SIZE. > > > > > > > > By this, khugepaged etc...can goes into inifinite reclaim loop > > > > if tasks in memcg are busy. > > > > > > > > After this patch > > > > - Hugepage allocation will fail if 1st trial of page reclaim fails. > > > > > > > > Changelog: > > > > - make changes small. removed renaming codes. > > > > > > > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > > > --- > > > > mm/memcontrol.c | 28 ++++++++++++++++++++++++---- > > > > 1 file changed, 24 insertions(+), 4 deletions(-) > > > > > > Was there something wrong with my oneline fix? > > > > > I thought your patch was against RHEL6. > > Sorry, this was a misunderstanding. All three patches I sent > yesterday were based on the latest mmotm. > I misunderstand your e-mail. > > > Really, there is no way to make this a beautiful fix. The way this > > > function is split up makes no sense, and the constant addition of more > > > and more flags just to correctly communicate with _one callsite_ > > > should be an obvious hint. > > > > > > > Your version has to depend on oom_check flag to work fine. > > I think it's complicated. > > I don't understand. We want to retry when batching fails, but not > when huge page charging fails. This is exactly what my patch does. > > This function has 3 steps: > > 1. charge > 2. reclaim > 3. handle out of memory > > Between all those steps, there are defined break-out points. Between > 1. and 2. there is the check for batching. Between 2. and 3. is the > check for whether we should OOM directly or let it be handled by the > caller. > > These break points make perferct sense, because when batching we want > to charge but not reclaim. With huge pages we want to charge, > rcelaim, but not OOM. This is straight-forward exactly what my > patches implement. Not by introducing new break points, but by fixing > those that are already there! > > I resent your patch because you mess up this logic by moving the break > point between 1. and 2. between 2. and 3. where it is not intuitive to > understand anymore. And your argument is that you don't want to trust > your own code to function correctly (oom_check). This is insane. > oom_check is not for contorlling 'how we retry'. But I'm getting tired at handling this breakage caused by THP. Please post your patch again, I'll ack all in the next week. Andrew, please pick up Johannes's patch when psoted. I hope all issues are fixed in the next week and we can go ahead, implenting dirty_ratio. Bye. -Kame
WARNING: multiple messages have this Message-ID (diff)
From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> To: Johannes Weiner <hannes@cmpxchg.org> Cc: "linux-mm@kvack.org" <linux-mm@kvack.org>, "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>, "nishimura@mxp.nes.nec.co.jp" <nishimura@mxp.nes.nec.co.jp>, "balbir@linux.vnet.ibm.com" <balbir@linux.vnet.ibm.com> Subject: Re: [BUGFIX][PATCH 2/4] memcg: fix charge path for THP and allow early retirement Date: Fri, 28 Jan 2011 18:16:53 +0900 [thread overview] Message-ID: <20110128181653.240c73a0.kamezawa.hiroyu@jp.fujitsu.com> (raw) In-Reply-To: <20110128090242.GF2213@cmpxchg.org> On Fri, 28 Jan 2011 10:02:42 +0100 Johannes Weiner <hannes@cmpxchg.org> wrote: > On Fri, Jan 28, 2011 at 05:14:47PM +0900, KAMEZAWA Hiroyuki wrote: > > On Fri, 28 Jan 2011 08:57:24 +0100 > > Johannes Weiner <hannes@cmpxchg.org> wrote: > > > > > On Fri, Jan 28, 2011 at 12:26:08PM +0900, KAMEZAWA Hiroyuki wrote: > > > > From: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > > > > > > > When THP is used, Hugepage size charge can happen. It's not handled > > > > correctly in mem_cgroup_do_charge(). For example, THP can fallback > > > > to small page allocation when HUGEPAGE allocation seems difficult > > > > or busy, but memory cgroup doesn't understand it and continue to > > > > try HUGEPAGE charging. And the worst thing is memory cgroup > > > > believes 'memory reclaim succeeded' if limit - usage > PAGE_SIZE. > > > > > > > > By this, khugepaged etc...can goes into inifinite reclaim loop > > > > if tasks in memcg are busy. > > > > > > > > After this patch > > > > - Hugepage allocation will fail if 1st trial of page reclaim fails. > > > > > > > > Changelog: > > > > - make changes small. removed renaming codes. > > > > > > > > Signed-off-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com> > > > > --- > > > > mm/memcontrol.c | 28 ++++++++++++++++++++++++---- > > > > 1 file changed, 24 insertions(+), 4 deletions(-) > > > > > > Was there something wrong with my oneline fix? > > > > > I thought your patch was against RHEL6. > > Sorry, this was a misunderstanding. All three patches I sent > yesterday were based on the latest mmotm. > I misunderstand your e-mail. > > > Really, there is no way to make this a beautiful fix. The way this > > > function is split up makes no sense, and the constant addition of more > > > and more flags just to correctly communicate with _one callsite_ > > > should be an obvious hint. > > > > > > > Your version has to depend on oom_check flag to work fine. > > I think it's complicated. > > I don't understand. We want to retry when batching fails, but not > when huge page charging fails. This is exactly what my patch does. > > This function has 3 steps: > > 1. charge > 2. reclaim > 3. handle out of memory > > Between all those steps, there are defined break-out points. Between > 1. and 2. there is the check for batching. Between 2. and 3. is the > check for whether we should OOM directly or let it be handled by the > caller. > > These break points make perferct sense, because when batching we want > to charge but not reclaim. With huge pages we want to charge, > rcelaim, but not OOM. This is straight-forward exactly what my > patches implement. Not by introducing new break points, but by fixing > those that are already there! > > I resent your patch because you mess up this logic by moving the break > point between 1. and 2. between 2. and 3. where it is not intuitive to > understand anymore. And your argument is that you don't want to trust > your own code to function correctly (oom_check). This is insane. > oom_check is not for contorlling 'how we retry'. But I'm getting tired at handling this breakage caused by THP. Please post your patch again, I'll ack all in the next week. Andrew, please pick up Johannes's patch when psoted. I hope all issues are fixed in the next week and we can go ahead, implenting dirty_ratio. Bye. -Kame -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Fight unfair telecom policy in Canada: sign http://dissolvethecrtc.ca/ Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2011-01-28 9:23 UTC|newest] Thread overview: 66+ messages / expand[flat|nested] mbox.gz Atom feed top 2011-01-28 3:22 [BUGFIX][PATCH 0/4] Fixes for memcg with THP KAMEZAWA Hiroyuki 2011-01-28 3:22 ` KAMEZAWA Hiroyuki 2011-01-28 3:24 ` [BUGFIX][PATCH 1/4] memcg: fix limit estimation at reclaim for hugepage KAMEZAWA Hiroyuki 2011-01-28 3:24 ` KAMEZAWA Hiroyuki 2011-01-28 4:40 ` Daisuke Nishimura 2011-01-28 4:40 ` Daisuke Nishimura 2011-01-28 4:49 ` KAMEZAWA Hiroyuki 2011-01-28 4:49 ` KAMEZAWA Hiroyuki 2011-01-28 4:58 ` KAMEZAWA Hiroyuki 2011-01-28 4:58 ` KAMEZAWA Hiroyuki 2011-01-28 5:36 ` Daisuke Nishimura 2011-01-28 5:36 ` Daisuke Nishimura 2011-01-28 8:04 ` Minchan Kim 2011-01-28 8:04 ` Minchan Kim 2011-01-28 8:17 ` Johannes Weiner 2011-01-28 8:17 ` Johannes Weiner 2011-01-28 8:25 ` Minchan Kim 2011-01-28 8:25 ` Minchan Kim 2011-01-28 8:36 ` KAMEZAWA Hiroyuki 2011-01-28 8:36 ` KAMEZAWA Hiroyuki 2011-01-30 2:26 ` Minchan Kim 2011-01-30 2:26 ` Minchan Kim 2011-01-28 8:41 ` Johannes Weiner 2011-01-28 8:41 ` Johannes Weiner 2011-01-28 8:24 ` KAMEZAWA Hiroyuki 2011-01-28 8:24 ` KAMEZAWA Hiroyuki 2011-01-28 8:37 ` Minchan Kim 2011-01-28 8:37 ` Minchan Kim 2011-01-28 7:52 ` Johannes Weiner 2011-01-28 7:52 ` Johannes Weiner 2011-01-28 8:06 ` KAMEZAWA Hiroyuki 2011-01-28 8:06 ` KAMEZAWA Hiroyuki 2011-01-28 3:26 ` [BUGFIX][PATCH 2/4] memcg: fix charge path for THP and allow early retirement KAMEZAWA Hiroyuki 2011-01-28 3:26 ` KAMEZAWA Hiroyuki 2011-01-28 5:37 ` Daisuke Nishimura 2011-01-28 5:37 ` Daisuke Nishimura 2011-01-28 7:57 ` Johannes Weiner 2011-01-28 7:57 ` Johannes Weiner 2011-01-28 8:14 ` KAMEZAWA Hiroyuki 2011-01-28 8:14 ` KAMEZAWA Hiroyuki 2011-01-28 9:02 ` Johannes Weiner 2011-01-28 9:02 ` Johannes Weiner 2011-01-28 9:16 ` KAMEZAWA Hiroyuki [this message] 2011-01-28 9:16 ` KAMEZAWA Hiroyuki 2011-01-28 3:27 ` [BUGFIX][PATCH 3/4] mecg: fix oom flag at THP charge KAMEZAWA Hiroyuki 2011-01-28 3:27 ` KAMEZAWA Hiroyuki 2011-01-28 5:39 ` Daisuke Nishimura 2011-01-28 5:39 ` Daisuke Nishimura 2011-01-28 5:50 ` KAMEZAWA Hiroyuki 2011-01-28 5:50 ` KAMEZAWA Hiroyuki 2011-01-28 8:02 ` Johannes Weiner 2011-01-28 8:02 ` Johannes Weiner 2011-01-28 8:21 ` KAMEZAWA Hiroyuki 2011-01-28 8:21 ` KAMEZAWA Hiroyuki 2011-01-31 7:41 ` Balbir Singh 2011-01-31 7:41 ` Balbir Singh 2011-01-28 3:28 ` [BUGFIX][PATCH 4/4] memcg: fix khugepaged should skip busy memcg KAMEZAWA Hiroyuki 2011-01-28 3:28 ` KAMEZAWA Hiroyuki 2011-01-28 8:20 ` Daisuke Nishimura 2011-01-28 8:20 ` Daisuke Nishimura 2011-01-28 8:30 ` KAMEZAWA Hiroyuki 2011-01-28 8:30 ` KAMEZAWA Hiroyuki 2011-01-29 12:47 ` [BUGFIX][PATCH 0/4] Fixes for memcg with THP Balbir Singh 2011-01-29 12:47 ` Balbir Singh 2011-01-30 23:55 ` KAMEZAWA Hiroyuki 2011-01-30 23:55 ` KAMEZAWA Hiroyuki
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20110128181653.240c73a0.kamezawa.hiroyu@jp.fujitsu.com \ --to=kamezawa.hiroyu@jp.fujitsu.com \ --cc=balbir@linux.vnet.ibm.com \ --cc=hannes@cmpxchg.org \ --cc=linux-kernel@vger.kernel.org \ --cc=linux-mm@kvack.org \ --cc=nishimura@mxp.nes.nec.co.jp \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.