From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1763601AbdDSNW0 (ORCPT ); Wed, 19 Apr 2017 09:22:26 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:46587 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1763374AbdDSNWX (ORCPT ); Wed, 19 Apr 2017 09:22:23 -0400 Date: Wed, 19 Apr 2017 06:22:13 -0700 From: "Paul E. McKenney" To: Christian Borntraeger Cc: Peter Zijlstra , linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com, bobby.prani@gmail.com, marc.zyngier@arm.com Subject: Re: [PATCH v2 tip/core/rcu 0/13] Miscellaneous fixes for 4.12 Reply-To: paulmck@linux.vnet.ibm.com References: <20170412165441.GA17149@linux.vnet.ibm.com> <20170417232714.GA19013@linux.vnet.ibm.com> <20170419112845.3rt3zlyzuzert647@hirez.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 x-cbid: 17041913-0044-0000-0000-0000030C1711 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006938; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000208; SDB=6.00849611; UDB=6.00419540; IPR=6.00628244; BA=6.00005304; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00015094; XFM=3.00000013; UTC=2017-04-19 13:22:20 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17041913-0045-0000-0000-0000073A1D21 Message-Id: <20170419132213.GA17362@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-04-19_11:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=2 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1703280000 definitions=main-1704190117 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Apr 19, 2017 at 01:48:08PM +0200, Christian Borntraeger wrote: > On 04/19/2017 01:28 PM, Peter Zijlstra wrote: > > > > So the thing Maz complained about is because KVM assumes > > synchronize_srcu() is 'free' when there is no srcu_read_lock() activity. > > This series 'breaks' that. > > Why is such a behaviour change not mentioned in the cover letter? > I could not find anything in the patch descriptions that would > indicate a slowdown. How much slower did it get? It was an 8x slowdown in boot time of a guest OS running UEFI, from five seconds to forty seconds. The fix restored the original boot time. Why didn't I report the slowdown in my cover letter? Because I didn't realize that I had created such a stupid bug! ;-) Why didn't my testing reveal the bug? Because in my rcutorture testing, the buggy code runs about as fast as the original, and the fixed new code runs about an order of magnitude faster. This is because rcutorture's performance statistics are mostly sensitive to throughput, while Marc's boot-time run is mostly sensitive to latency. > But indeed, there are several places at KVM startup which have been > reworked to srcu since normal rcu was too slow for several usecases. > (Mostly registering devices and related data structures at startup, > basically the qemu/kvm coldplug interaction) And here is the patch that restored Marc's boot speed. It simply changes the original (buggy) fixed delay for no delay in the expedited case and the same fixed delay in the non-expedited case. Thanx, Paul ------------------------------------------------------------------------ commit 66ae176ab33dd3afa0b944d149fe8240e65743f9 Author: Paul E. McKenney Date: Tue Apr 18 10:28:31 2017 -0700 srcu: Expedite srcu_schedule_cbs_snp() callback invocation Although Tree SRCU does reduce delays when there is at least one synchronize_srcu_expedited() invocation pending, srcu_schedule_cbs_snp() still waits for SRCU_INTERVAL before invoking callbacks. Since synchronize_srcu_expedited() now posts a callback and waits for that callback to do a wakeup, this destroys the expedited nature of synchronize_srcu_expedited(). This destruction became apparent to Marc Zyngier in the guise of a guest-OS bootup slowdown from five seconds to no fewer than forty seconds. This commit therefore invokes callbacks immediately at the end of the grace period when there is at least one synchronize_srcu_expedited() invocation pending. This brought Marc's guest-OS bootup times back into the realm of reason. Reported-by: Marc Zyngier Signed-off-by: Paul E. McKenney Tested-by: Marc Zyngier diff --git a/kernel/rcu/srcutree.c b/kernel/rcu/srcutree.c index f9c684d79faa..e11b89a363f7 100644 --- a/kernel/rcu/srcutree.c +++ b/kernel/rcu/srcutree.c @@ -442,7 +442,8 @@ static void srcu_schedule_cbs_snp(struct srcu_struct *sp, struct srcu_node *snp) int cpu; for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) - srcu_schedule_cbs_sdp(per_cpu_ptr(sp->sda, cpu), SRCU_INTERVAL); + srcu_schedule_cbs_sdp(per_cpu_ptr(sp->sda, cpu), + atomic_read(&sp->srcu_exp_cnt) ? 0 : SRCU_INTERVAL); } /*