From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1764940AbdDSOry (ORCPT ); Wed, 19 Apr 2017 10:47:54 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:52007 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S935487AbdDSOrk (ORCPT ); Wed, 19 Apr 2017 10:47:40 -0400 Date: Wed, 19 Apr 2017 07:47:30 -0700 From: "Paul E. McKenney" To: Peter Zijlstra Cc: Christian Borntraeger , linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com, marc.zyngier@arm.com Subject: Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Reply-To: paulmck@linux.vnet.ibm.com References: <20170412165441.GA17149@linux.vnet.ibm.com> <20170417232714.GA19013@linux.vnet.ibm.com> <20170419112845.3rt3zlyzuzert647@hirez.programming.kicks-ass.net> <20170419120847.GB3029@worktop.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170419120847.GB3029@worktop.programming.kicks-ass.net> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17041914-0044-0000-0000-0000030C3FCE X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006939; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000208; SDB=6.00849639; UDB=6.00419557; IPR=6.00628272; BA=6.00005304; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015094; XFM=3.00000013; UTC=2017-04-19 14:47:36 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17041914-0045-0000-0000-0000073A45DB Message-Id: <20170419144730.GA20845@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-19_12:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704190125 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 19, 2017 at 02:08:47PM +0200, Peter Zijlstra wrote: > On Wed, Apr 19, 2017 at 01:48:08PM +0200, Christian Borntraeger wrote: > > On 04/19/2017 01:28 PM, Peter Zijlstra wrote: > > > > > > So the thing Maz complained about is because KVM assumes > > > synchronize_srcu() is 'free' when there is no srcu_read_lock() activity. > > > This series 'breaks' that. > > > > Why is such a behaviour change not mentioned in the cover letter? > > I could not find anything in the patch descriptions that would > > indicate a slowdown. How much slower did it get? > > > > But indeed, there are several places at KVM startup which have been > > reworked to srcu since normal rcu was too slow for several usecases. > > (Mostly registering devices and related data structures at startup, > > basically the qemu/kvm coldplug interaction) > > I suspect Paul is not considering this a 'normal' RCU feature, and > therefore didn't think about changing this. > > I know I was fairly surprised by this requirement when I ran into it; > and only accidentally remembered it now that maz complained. Indeed -- the natural thing to have done back when KVM's scalability was first being worked on would have been to simply change synchronize_rcu() to synchronize_rcu_expedited(). However, at that time, these things did try_stop_cpus() and the like, which was really bad for latency. Moving to SRCU avoided this problem. Of course, now that KVM uses SRCU, why change unless there is a problem? Besides, I vaguely recall some KVM cases where srcu_read_lock() is used from CPUs that look to be idle or offline from RCU's perspective, and that sort of thing only works for SRCU. Which reminds me... The RCU expedited primitives have been completely rewritten since then, and no longer use try_stop_cpus(), no longer disturb idle CPUs, and no longer disturb nohz_full CPUs running in userspace. In addition, there is the rcupdate.rcu_normal kernel boot paramter for those who want to completely avoid RCU expedited primitives. So it seems to me to be time for the patch below. Thoughts? Thanx, Paul ------------------------------------------------------------------------ commit 333d383fad42b4bdef3d27d91e940a6eafed3f91 Author: Paul E. McKenney Date: Wed Apr 19 07:37:45 2017 -0700 checkpatch: Remove checks for expedited grace periods There was a time when the expedited grace-period primitives (synchronize_rcu_expedited(), synchronize_rcu_bh_expedited(), and synchronize_sched_expedited()) used rather antisocial kernel facilities like try_stop_cpus(). However, they have since been housebroken to use only single-CPU IPIs, and typically cause less disturbance than a scheduling-clock interrupt. Furthermore, this disturbance can be eliminated entirely using NO_HZ_FULL on the one hand or the rcupdate.rcu_normal boot parameter on the other. This commit therefore removes checkpatch's complaints about use of the expedited RCU primitives. Signed-off-by: Paul E. McKenney diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl index baa3c7be04ad..64bf2a091368 100755 --- a/scripts/checkpatch.pl +++ b/scripts/checkpatch.pl @@ -5511,23 +5511,6 @@ sub process { } } -# Check for expedited grace periods that interrupt non-idle non-nohz -# online CPUs. These expedited can therefore degrade real-time response -# if used carelessly, and should be avoided where not absolutely -# needed. It is always OK to use synchronize_rcu_expedited() and -# synchronize_sched_expedited() at boot time (before real-time applications -# start) and in error situations where real-time response is compromised in -# any case. Note that synchronize_srcu_expedited() does -not- interrupt -# other CPUs, so don't warn on uses of synchronize_srcu_expedited(). -# Of course, nothing comes for free, and srcu_read_lock() and -# srcu_read_unlock() do contain full memory barriers in payment for -# synchronize_srcu_expedited() non-interruption properties. - if ($line =~ /\b(synchronize_rcu_expedited|synchronize_sched_expedited)\(/) { - WARN("EXPEDITED_RCU_GRACE_PERIOD", - "expedited RCU grace periods should be avoided where they can degrade real-time response\n" . $herecurr); - - } - # check of hardware specific defines if ($line =~ m@^.\s*\#\s*if.*\b(__i386__|__powerpc64__|__sun__|__s390x__)\b@ && $realfile !~ m@include/asm-@) { CHK("ARCH_DEFINES",