From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755004AbdDMRbI (ORCPT ); Thu, 13 Apr 2017 13:31:08 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:50378 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1754775AbdDMRbG (ORCPT ); Thu, 13 Apr 2017 13:31:06 -0400 Date: Thu, 13 Apr 2017 10:31:00 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com Subject: Re: [PATCH tip/core/rcu 04/13] rcu: Make RCU_FANOUT_LEAF help text more explicit about skew_tick Reply-To: paulmck@linux.vnet.ibm.com References: <20170412165441.GA17149@linux.vnet.ibm.com> <1492016149-18834-4-git-send-email-paulmck@linux.vnet.ibm.com> <20170413091535.r6iw7s3pc2znvl6b@hirez.programming.kicks-ass.net> <20170413160332.GZ3956@linux.vnet.ibm.com> <20170413161948.ymvzlzhporgmldvn@hirez.programming.kicks-ass.net> <20170413165516.GI3956@linux.vnet.ibm.com> <20170413170434.xk4zq3p75pu3ubxw@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170413170434.xk4zq3p75pu3ubxw@hirez.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17041317-0024-0000-0000-0000024678C4 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006930; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000208; SDB=6.00847080; UDB=6.00417894; IPR=6.00625514; BA=6.00005288; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015033; XFM=3.00000013; UTC=2017-04-13 17:31:03 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17041317-0025-0000-0000-0000435959D1 Message-Id: <20170413173100.GL3956@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-13_11:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=1 spamscore=1 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1704130146 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Apr 13, 2017 at 07:04:34PM +0200, Peter Zijlstra wrote: > On Thu, Apr 13, 2017 at 09:55:16AM -0700, Paul E. McKenney wrote: > > > > To avoid people tuning huge machines having to wait for me to give > > > > them an answer as to why they are suffering lock contention after > > > > cranking up the value of RCU_FANOUT_LEAF. > > So is there a good reason to increase FANOUT_LEAF ? Increasing it reduces the number of rcu_node structures, and thus the number of cache misses during grace-period initialization and cleanup. This has proven necessary in the past on large machines having long memory latencies. And there are starting to be some pretty big machines running in production, and even for typical commerical workloads. > > > > Or am I missing your point? > > > > > > Your answer should be: don't do that then. Not provide them a shady work > > > around. > > > > > > tick skew isn't pretty and has other problems (there's a reason its not > > > on by default). You're then doing two things you shouldn't. > > > > The tick skew problem that I know of is energy efficiency for light > > workloads. This doesn't normally apply to the large heavily loaded > > systems on which people skew ticks. > > > > So what are the other problems? > > If the jiffy updater bounces between CPUs (as is not uncommon) the > duration of the jiffy becomes an average (I think we fixed it where it > could go too fast by always jumping to a CPU which has a short jiffy, > but I'm not sure). > > This further complicates some of the jiffy based loops (which arguably > should go away anyway). Fair enough. And I am OK with jiffies going away as long as there is a low-overhead rough-and-ready timing mechanism replacing it. > And I have vague memories of it actually causing lock contention, but > I've forgotten how that worked. That is a new one on me. I can easily see how not skewing ticks could cause serious lock contention, but am missing how skewed ticks would do so. Thanx, Paul