From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1750775AbdAXRH3 (ORCPT ); Tue, 24 Jan 2017 12:07:29 -0500 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:50347 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750716AbdAXRH1 (ORCPT ); Tue, 24 Jan 2017 12:07:27 -0500 Date: Tue, 24 Jan 2017 09:07:19 -0800 From: "Paul E. McKenney" To: Lance Roy Cc: bobby.prani@gmail.com, linux-kernel@vger.kernel.org, mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@efficios.com, josh@joshtriplett.org, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, dhowells@redhat.com, edumazet@google.com, dvhart@linux.intel.com, fweisbec@gmail.com, oleg@redhat.com Subject: Re: [PATCH] srcu: Implement more-efficient reader counts Reply-To: paulmck@linux.vnet.ibm.com References: <20170123133359.2d69756a@gmail.com> <20170123213518.7200-1-ldr709@gmail.com> <20170124004252.GZ28085@linux.vnet.ibm.com> <20170123165334.6873e47f@gmail.com> <20170124015715.GD28085@linux.vnet.ibm.com> <20170123192645.69507b2f@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20170123192645.69507b2f@gmail.com> User-Agent: Mutt/1.5.21 (2010-09-15) X-TM-AS-GCONF: 00 X-Content-Scanned: Fidelis XPS MAILER x-cbid: 17012417-0008-0000-0000-000007029F06 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00006490; HX=3.00000240; KW=3.00000007; PH=3.00000004; SC=3.00000200; SDB=6.00812088; UDB=6.00396007; IPR=6.00589504; BA=6.00005085; NDR=6.00000001; ZLA=6.00000005; ZF=6.00000009; ZB=6.00000000; ZP=6.00000000; ZH=6.00000000; ZU=6.00000002; MB=3.00014028; XFM=3.00000011; UTC=2017-01-24 17:07:23 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17012417-0009-0000-0000-00003F4689D1 Message-Id: <20170124170719.GQ28085@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-01-24_13:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1612050000 definitions=main-1701240111 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Jan 23, 2017 at 07:26:45PM -0800, Lance Roy wrote: > > Yeah, we did have this same conversation awhile back, didn't we? > > > > Back then, did I think to ask if this could be minimized or even prevented > > by adding memory barriers appropriately? ;-) > > > > Thanx, Paul > > Yes, it can be fixed by adding a memory barrier after incrementing ->completed > inside srcu_flip(). The upper limit on NR_CPUS turns out to be more complicated > than this, as it needs to deal with highly nested read side critical sections > mixed with the critical section loops, but only the one memory barrier should > be necessary. Something like this, then? Thanx, Paul ------------------------------------------------------------------------ commit 35be9e413dde662fc9661352e595105ac4b0b167 Author: Paul E. McKenney Date: Tue Jan 24 08:51:34 2017 -0800 srcu: Reduce probability of SRCU ->unlock_count[] counter overflow Because there are no memory barriers between the srcu_flip() ->completed increment and the summation of the read-side ->unlock_count[] counters, both the compiler and the CPU can reorder the summation with the ->completed increment. If the updater is preempted long enough during this process, the read-side counters could overflow, resulting in a too-short grace period. This commit therefore adds a memory barrier just after the ->completed increment, ensuring that if the summation misses an increment of ->unlock_count[] from __srcu_read_unlock(), the next __srcu_read_lock() will see the new value of ->completed, thus bounding the number of ->unlock_count[] increments that can be missed to NR_CPUS. The actual overflow computation is more complex due to the possibility of nesting of __srcu_read_lock(). Reported-by: Lance Roy Signed-off-by: Paul E. McKenney diff --git a/kernel/rcu/srcu.c b/kernel/rcu/srcu.c index d3378ceb9762..aefe3ab20a6a 100644 --- a/kernel/rcu/srcu.c +++ b/kernel/rcu/srcu.c @@ -337,7 +337,16 @@ static bool try_check_zero(struct srcu_struct *sp, int idx, int trycount) */ static void srcu_flip(struct srcu_struct *sp) { - sp->completed++; + WRITE_ONCE(sp->completed, sp->completed + 1); + + /* + * Ensure that if the updater misses an __srcu_read_unlock() + * increment, that task's next __srcu_read_lock() will see the + * above counter update. Note that both this memory barrier + * and the one in srcu_readers_active_idx_check() provide the + * guarantee for __srcu_read_lock(). + */ + smp_mb(); /* D */ /* Pairs with C. */ } /*