From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1759338Ab2IETuF (ORCPT ); Wed, 5 Sep 2012 15:50:05 -0400 Received: from e39.co.us.ibm.com ([32.97.110.160]:50392 "EHLO e39.co.us.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1759143Ab2IETuC (ORCPT ); Wed, 5 Sep 2012 15:50:02 -0400 Date: Wed, 5 Sep 2012 12:49:18 -0700 From: "Paul E. McKenney" To: Josh Triplett Cc: linux-kernel@vger.kernel.org, mingo@elte.hu, laijs@cn.fujitsu.com, dipankar@in.ibm.com, akpm@linux-foundation.org, mathieu.desnoyers@polymtl.ca, niv@us.ibm.com, tglx@linutronix.de, peterz@infradead.org, rostedt@goodmis.org, Valdis.Kletnieks@vt.edu, dhowells@redhat.com, eric.dumazet@gmail.com, darren@dvhart.com, fweisbec@gmail.com, sbw@mit.edu, patches@linaro.org Subject: Re: [PATCH tip/core/rcu 16/23] rcu: Prevent initialization-time quiescent-state race Message-ID: <20120905194918.GQ3308@linux.vnet.ibm.com> Reply-To: paulmck@linux.vnet.ibm.com References: <20120830181811.GA29154@linux.vnet.ibm.com> <1346350718-30937-1-git-send-email-paulmck@linux.vnet.ibm.com> <1346350718-30937-16-git-send-email-paulmck@linux.vnet.ibm.com> <20120903093742.GE5574@leaf> <20120905181920.GN3308@linux.vnet.ibm.com> <20120905185534.GA4976@jtriplet-mobl1> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20120905185534.GA4976@jtriplet-mobl1> User-Agent: Mutt/1.5.21 (2010-09-15) X-Content-Scanned: Fidelis XPS MAILER x-cbid: 12090519-4242-0000-0000-000002CB844C Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Sep 05, 2012 at 11:55:34AM -0700, Josh Triplett wrote: > On Wed, Sep 05, 2012 at 11:19:20AM -0700, Paul E. McKenney wrote: > > On Mon, Sep 03, 2012 at 02:37:42AM -0700, Josh Triplett wrote: > > > On Thu, Aug 30, 2012 at 11:18:31AM -0700, Paul E. McKenney wrote: > > > > From: "Paul E. McKenney" > > > > > > > > Now the the grace-period initialization procedure is preemptible, it is > > > > subject to the following race on systems whose rcu_node tree contains > > > > more than one node: > > > > > > > > 1. CPU 31 starts initializing the grace period, including the > > > > first leaf rcu_node structures, and is then preempted. > > > > > > > > 2. CPU 0 refers to the first leaf rcu_node structure, and notes > > > > that a new grace period has started. It passes through a > > > > quiescent state shortly thereafter, and informs the RCU core > > > > of this rite of passage. > > > > > > > > 3. CPU 0 enters an RCU read-side critical section, acquiring > > > > a pointer to an RCU-protected data item. > > > > > > > > 4. CPU 31 removes the data item referenced by CPU 0 from the > > > > data structure, and registers an RCU callback in order to > > > > free it. > > > > > > > > 5. CPU 31 resumes initializing the grace period, including its > > > > own rcu_node structure. In invokes rcu_start_gp_per_cpu(), > > > > which advances all callbacks, including the one registered > > > > in #4 above, to be handled by the current grace period. > > > > > > > > 6. The remaining CPUs pass through quiescent states and inform > > > > the RCU core, but CPU 0 remains in its RCU read-side critical > > > > section, still referencing the now-removed data item. > > > > > > > > 7. The grace period completes and all the callbacks are invoked, > > > > including the one that frees the data item that CPU 0 is still > > > > referencing. Oops!!! > > > > > > > > This commit therefore moves the callback handling to precede initialization > > > > of any of the rcu_node structures, thus avoiding this race. > > > > > > I don't think it makes sense to introduce and subsequently fix a race in > > > the same patch series. :) > > > > > > Could you squash this patch into the one moving grace-period > > > initialization into a kthread? > > > > I tried that, and got a surprisingly large set of conflicts. Ah, OK, > > the problem is that breaking up rcu_gp_kthread() into subfunctions > > did enough code motion to defeat straightforward rebasing. Is there > > some way to tell "git rebase" about such code motion, or would this > > need to be carried out carefully by hand? > > To the extent rebase knows how to handle that, I think it does so > automatically as part of merge attempts. Fortunately, in this case, the > change consists of moving two lines of code and their attached comment, > which seems easy enough to change in the original code; you'll then get > a conflict on the commit that moves the newly fixed code (easily > resolved by moving the change to the new code), and conflicts on any > changes next to the change in the new code (hopefully handled by > three-way merge, and if not then easily fixed by keeping the new lines). Good point, perhaps if I do the code movement manually and use multiple rebases it will go more easily. Thanx, Paul