From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756305Ab0DNQXL (ORCPT ); Wed, 14 Apr 2010 12:23:11 -0400 Received: from smtp-out.google.com ([74.125.121.35]:56031 "EHLO smtp-out.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756275Ab0DNQXI convert rfc822-to-8bit (ORCPT ); Wed, 14 Apr 2010 12:23:08 -0400 DomainKey-Signature: a=rsa-sha1; s=beta; d=google.com; c=nofws; q=dns; h=mime-version:in-reply-to:references:from:date:message-id: subject:to:cc:content-type:content-transfer-encoding:x-system-of-record; b=HYjkuJxnJO8GmV3Hrdlmi8zT20Fbh9bWNU4F1NPu64qXWHmjypE9tUeai5U40miDJ CknoPHm2Iv1DRC+tdjeEQ== MIME-Version: 1.0 In-Reply-To: <20100414182904.2f72a63d.kamezawa.hiroyu@jp.fujitsu.com> References: <1268609202-15581-1-git-send-email-arighi@develer.com> <20100318085411.834e1e46.kamezawa.hiroyu@jp.fujitsu.com> <20100318041944.GA18054@balbir.in.ibm.com> <20100318133527.420b2f25.kamezawa.hiroyu@jp.fujitsu.com> <20100318162855.GG18054@balbir.in.ibm.com> <20100319102332.f1d81c8d.kamezawa.hiroyu@jp.fujitsu.com> <20100319024039.GH18054@balbir.in.ibm.com> <20100319120049.3dbf8440.kamezawa.hiroyu@jp.fujitsu.com> <20100414182904.2f72a63d.kamezawa.hiroyu@jp.fujitsu.com> From: Greg Thelen Date: Wed, 14 Apr 2010 09:22:41 -0700 Message-ID: Subject: Re: [PATCH -mmotm 1/5] memcg: disable irq at page cgroup lock To: KAMEZAWA Hiroyuki Cc: balbir@linux.vnet.ibm.com, Andrea Righi , Daisuke Nishimura , Vivek Goyal , Peter Zijlstra , Trond Myklebust , Suleiman Souhlal , "Kirill A. Shutemov" , Andrew Morton , containers@lists.linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8BIT X-System-Of-Record: true Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 14, 2010 at 2:29 AM, KAMEZAWA Hiroyuki wrote: > On Tue, 13 Apr 2010 23:55:12 -0700 > Greg Thelen wrote: >> On Thu, Mar 18, 2010 at 8:00 PM, KAMEZAWA Hiroyuki wrote: >> > On Fri, 19 Mar 2010 08:10:39 +0530 >> > Balbir Singh wrote: >> >> * KAMEZAWA Hiroyuki [2010-03-19 10:23:32]: >> >> >> >> > On Thu, 18 Mar 2010 21:58:55 +0530 >> >> > Balbir Singh wrote: >> >> > >> >> > > * KAMEZAWA Hiroyuki [2010-03-18 13:35:27]: >> >> > >> >> > > > Then, no probelm. It's ok to add mem_cgroup_udpate_stat() indpendent from >> >> > > > mem_cgroup_update_file_mapped(). The look may be messy but it's not your >> >> > > > fault. But please write "why add new function" to patch description. >> >> > > > >> >> > > > I'm sorry for wasting your time. >> >> > > >> >> > > Do we need to go down this route? We could check the stat and do the >> >> > > correct thing. In case of FILE_MAPPED, always grab page_cgroup_lock >> >> > > and for others potentially look at trylock. It is OK for different >> >> > > stats to be protected via different locks. >> >> > > >> >> > >> >> > I _don't_ want to see a mixture of spinlock and trylock in a function. >> >> > >> >> >> >> A well documented well written function can help. The other thing is to >> >> of-course solve this correctly by introducing different locking around >> >> the statistics. Are you suggesting the later? >> >> >> > >> > No. As I wrote. >> >        - don't modify codes around FILE_MAPPED in this series. >> >        - add a new functions for new statistics >> > Then, >> >        - think about clean up later, after we confirm all things work as expected. >> >> I have ported Andrea Righi's memcg dirty page accounting patches to latest >> mmtom-2010-04-05-16-09.  In doing so I have to address this locking issue.  Does >> the following look good?  I will (of course) submit the entire patch for review, >> but I wanted make sure I was aiming in the right direction. >> >> void mem_cgroup_update_page_stat(struct page *page, >>                       enum mem_cgroup_write_page_stat_item idx, bool charge) >> { >>       static int seq; >>       struct page_cgroup *pc; >> >>       if (mem_cgroup_disabled()) >>               return; >>       pc = lookup_page_cgroup(page); >>       if (!pc || mem_cgroup_is_root(pc->mem_cgroup)) >>               return; >> >>       /* >>        * This routine does not disable irq when updating stats.  So it is >>        * possible that a stat update from within interrupt routine, could >>        * deadlock.  Use trylock_page_cgroup() to avoid such deadlock.  This >>        * makes the memcg counters fuzzy.  More complicated, or lower >>        * performing locking solutions avoid this fuzziness, but are not >>        * currently needed. >>        */ >>       if (irqs_disabled()) { >>               if (! trylock_page_cgroup(pc)) >>                       return; >>       } else >>               lock_page_cgroup(pc); >> > > I prefer trylock_page_cgroup() always. What is your reason for preferring trylock_page_cgroup()? I assume it's for code simplicity, but I wanted to check. I had though about using trylock_page_cgroup() always, but I think that would make file_mapped accounting even more fuzzy that it already it is. I was trying to retain the current accuracy of file_mapped and only make new counters, like writeback/dirty/etc (those obtained in interrupt), fuzzy. > I have another idea fixing this up _later_. (But I want to start from simple one.) > > My rough idea is following.  Similar to your idea which you gave me before. Hi Kame-san, I like the general approach. The code I previously gave you appears to work and is faster than non-root memcgs using mmotm due to mostly being lockless. > == > DEFINE_PERCPU(account_move_ongoing); What's the reason for having a per-cpu account_move_ongoing flag? Would a single system-wide global be sufficient? I assume the majority of the time this value will not be changing because accounting moves are rare. Perhaps all of the per-cpu variables are packed within a per-cpu cacheline making accessing it more likely to be local, but I'm not sure if this is true. -- Greg From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail138.messagelabs.com (mail138.messagelabs.com [216.82.249.35]) by kanga.kvack.org (Postfix) with ESMTP id 14A296B01F2 for ; Wed, 14 Apr 2010 12:23:09 -0400 (EDT) Received: from kpbe13.cbf.corp.google.com (kpbe13.cbf.corp.google.com [172.25.105.77]) by smtp-out.google.com with ESMTP id o3EGN3qh012086 for ; Wed, 14 Apr 2010 18:23:05 +0200 Received: from qyk1 (qyk1.prod.google.com [10.241.83.129]) by kpbe13.cbf.corp.google.com with ESMTP id o3EGMuZW017239 for ; Wed, 14 Apr 2010 11:23:02 -0500 Received: by qyk1 with SMTP id 1so354619qyk.15 for ; Wed, 14 Apr 2010 09:23:01 -0700 (PDT) MIME-Version: 1.0 In-Reply-To: <20100414182904.2f72a63d.kamezawa.hiroyu@jp.fujitsu.com> References: <1268609202-15581-1-git-send-email-arighi@develer.com> <20100318085411.834e1e46.kamezawa.hiroyu@jp.fujitsu.com> <20100318041944.GA18054@balbir.in.ibm.com> <20100318133527.420b2f25.kamezawa.hiroyu@jp.fujitsu.com> <20100318162855.GG18054@balbir.in.ibm.com> <20100319102332.f1d81c8d.kamezawa.hiroyu@jp.fujitsu.com> <20100319024039.GH18054@balbir.in.ibm.com> <20100319120049.3dbf8440.kamezawa.hiroyu@jp.fujitsu.com> <20100414182904.2f72a63d.kamezawa.hiroyu@jp.fujitsu.com> From: Greg Thelen Date: Wed, 14 Apr 2010 09:22:41 -0700 Message-ID: Subject: Re: [PATCH -mmotm 1/5] memcg: disable irq at page cgroup lock Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Sender: owner-linux-mm@kvack.org To: KAMEZAWA Hiroyuki Cc: balbir@linux.vnet.ibm.com, Andrea Righi , Daisuke Nishimura , Vivek Goyal , Peter Zijlstra , Trond Myklebust , Suleiman Souhlal , "Kirill A. Shutemov" , Andrew Morton , containers@lists.linux-foundation.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org List-ID: On Wed, Apr 14, 2010 at 2:29 AM, KAMEZAWA Hiroyuki wrote: > On Tue, 13 Apr 2010 23:55:12 -0700 > Greg Thelen wrote: >> On Thu, Mar 18, 2010 at 8:00 PM, KAMEZAWA Hiroyuki wrote: >> > On Fri, 19 Mar 2010 08:10:39 +0530 >> > Balbir Singh wrote: >> >> * KAMEZAWA Hiroyuki [2010-03-19 10:2= 3:32]: >> >> >> >> > On Thu, 18 Mar 2010 21:58:55 +0530 >> >> > Balbir Singh wrote: >> >> > >> >> > > * KAMEZAWA Hiroyuki [2010-03-18 = 13:35:27]: >> >> > >> >> > > > Then, no probelm. It's ok to add mem_cgroup_udpate_stat() indpe= ndent from >> >> > > > mem_cgroup_update_file_mapped(). The look may be messy but it's= not your >> >> > > > fault. But please write "why add new function" to patch descrip= tion. >> >> > > > >> >> > > > I'm sorry for wasting your time. >> >> > > >> >> > > Do we need to go down this route? We could check the stat and do = the >> >> > > correct thing. In case of FILE_MAPPED, always grab page_cgroup_lo= ck >> >> > > and for others potentially look at trylock. It is OK for differen= t >> >> > > stats to be protected via different locks. >> >> > > >> >> > >> >> > I _don't_ want to see a mixture of spinlock and trylock in a functi= on. >> >> > >> >> >> >> A well documented well written function can help. The other thing is = to >> >> of-course solve this correctly by introducing different locking aroun= d >> >> the statistics. Are you suggesting the later? >> >> >> > >> > No. As I wrote. >> > =A0 =A0 =A0 =A0- don't modify codes around FILE_MAPPED in this series. >> > =A0 =A0 =A0 =A0- add a new functions for new statistics >> > Then, >> > =A0 =A0 =A0 =A0- think about clean up later, after we confirm all thin= gs work as expected. >> >> I have ported Andrea Righi's memcg dirty page accounting patches to late= st >> mmtom-2010-04-05-16-09. =A0In doing so I have to address this locking is= sue. =A0Does >> the following look good? =A0I will (of course) submit the entire patch f= or review, >> but I wanted make sure I was aiming in the right direction. >> >> void mem_cgroup_update_page_stat(struct page *page, >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 enum mem_cgroup_write_page_s= tat_item idx, bool charge) >> { >> =A0 =A0 =A0 static int seq; >> =A0 =A0 =A0 struct page_cgroup *pc; >> >> =A0 =A0 =A0 if (mem_cgroup_disabled()) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return; >> =A0 =A0 =A0 pc =3D lookup_page_cgroup(page); >> =A0 =A0 =A0 if (!pc || mem_cgroup_is_root(pc->mem_cgroup)) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 return; >> >> =A0 =A0 =A0 /* >> =A0 =A0 =A0 =A0* This routine does not disable irq when updating stats. = =A0So it is >> =A0 =A0 =A0 =A0* possible that a stat update from within interrupt routi= ne, could >> =A0 =A0 =A0 =A0* deadlock. =A0Use trylock_page_cgroup() to avoid such de= adlock. =A0This >> =A0 =A0 =A0 =A0* makes the memcg counters fuzzy. =A0More complicated, or= lower >> =A0 =A0 =A0 =A0* performing locking solutions avoid this fuzziness, but = are not >> =A0 =A0 =A0 =A0* currently needed. >> =A0 =A0 =A0 =A0*/ >> =A0 =A0 =A0 if (irqs_disabled()) { >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 if (! trylock_page_cgroup(pc)) >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 return; >> =A0 =A0 =A0 } else >> =A0 =A0 =A0 =A0 =A0 =A0 =A0 lock_page_cgroup(pc); >> > > I prefer trylock_page_cgroup() always. What is your reason for preferring trylock_page_cgroup()? I assume it's for code simplicity, but I wanted to check. I had though about using trylock_page_cgroup() always, but I think that would make file_mapped accounting even more fuzzy that it already it is. I was trying to retain the current accuracy of file_mapped and only make new counters, like writeback/dirty/etc (those obtained in interrupt), fuzzy. > I have another idea fixing this up _later_. (But I want to start from sim= ple one.) > > My rough idea is following. =A0Similar to your idea which you gave me bef= ore. Hi Kame-san, I like the general approach. The code I previously gave you appears to work and is faster than non-root memcgs using mmotm due to mostly being lockless. > =3D=3D > DEFINE_PERCPU(account_move_ongoing); What's the reason for having a per-cpu account_move_ongoing flag? Would a single system-wide global be sufficient? I assume the majority of the time this value will not be changing because accounting moves are rare. Perhaps all of the per-cpu variables are packed within a per-cpu cacheline making accessing it more likely to be local, but I'm not sure if this is true. -- Greg -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to majordomo@kvack.org. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: email@kvack.org