From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1751737AbdG0Ola (ORCPT <rfc822;w@1wt.eu>);
        Thu, 27 Jul 2017 10:41:30 -0400
Received: from foss.arm.com ([217.140.101.70]:47590 "EHLO foss.arm.com"
        rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP
        id S1751653AbdG0Ol3 (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Thu, 27 Jul 2017 10:41:29 -0400
Date: Thu, 27 Jul 2017 15:41:36 +0100
From: Will Deacon <will.deacon@arm.com>
To: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
Cc: Boqun Feng <boqun.feng@gmail.com>,
        Peter Zijlstra <peterz@infradead.org>, linux-kernel@vger.kernel.org,
        mingo@kernel.org, jiangshanlai@gmail.com, dipankar@in.ibm.com,
        akpm@linux-foundation.org, mathieu.desnoyers@efficios.com,
        josh@joshtriplett.org, tglx@linutronix.de, rostedt@goodmis.org,
        dhowells@redhat.com, edumazet@google.com, fweisbec@gmail.com,
        oleg@redhat.com
Subject: Re: [PATCH tip/core/rcu 4/5] sys_membarrier: Add expedited option
Message-ID: <20170727144136.GD20746@arm.com>
References: <20170725215510.GD28975@worktop>
 <20170725235936.GC3730@linux.vnet.ibm.com>
 <20170726074128.ybb3e4flnjkrpi74@hirez.programming.kicks-ass.net>
 <20170726154110.GN3730@linux.vnet.ibm.com>
 <20170727083003.ivb2fr47vepa2g6t@hirez.programming.kicks-ass.net>
 <20170727130816.GP3730@linux.vnet.ibm.com>
 <20170727135551.tg7cduw22krjzbvt@tardis>
 <20170727141633.GT3730@linux.vnet.ibm.com>
 <20170727142955.bb2bdgbjcm745xwo@tardis>
 <20170727143658.GW3730@linux.vnet.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20170727143658.GW3730@linux.vnet.ibm.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Jul 27, 2017 at 07:36:58AM -0700, Paul E. McKenney wrote:
> On Thu, Jul 27, 2017 at 10:29:55PM +0800, Boqun Feng wrote:
> > On Thu, Jul 27, 2017 at 07:16:33AM -0700, Paul E. McKenney wrote:
> > > On Thu, Jul 27, 2017 at 09:55:51PM +0800, Boqun Feng wrote:
> > > > Hi Paul,
> > > > 
> > > > I have a side question out of curiosity:
> > > > 
> > > > How does synchronize_sched() work properly for sys_membarrier()?
> > > > 
> > > > sys_membarrier() requires every other CPU does a smp_mb() before it
> > > > returns, and I know synchronize_sched() will wait until all CPUs running
> > > > a kernel thread do a context-switch, which has a smp_mb(). However, I
> > > > believe sched flavor RCU treat CPU running a user thread as a quiesent
> > > > state, so synchronize_sched() could return without that CPU does a
> > > > context switch. 
> > > > 
> > > > So why could we use synchronize_sched() for sys_membarrier()?
> > > > 
> > > > In particular, could the following happens?
> > > > 
> > > > 	CPU 0:				CPU 1:
> > > > 	=========================	==========================
> > > > 	<in user space>			<in user space>
> > > > 					{read Y}(reordered) <------------------------------+
> > > > 	store Y;			                                                   |
> > > > 					read X; --------------------------------------+    |
> > > > 	sys_membarrier():		<timer interrupt>                             |    |
> > > > 	  synchronize_sched();		  update_process_times(user): //user == true  |    |
> > > > 	  				    rcu_check_callbacks(usr):                 |    |
> > > > 					      if (user || ..) {                       |    |
> > > > 					        rcu_sched_qs()                        |    |
> > > > 						...                                   |    |
> > > > 						<report quesient state in softirq>    |    |
> > > 
> > > The reporting of the quiescent state will acquire the leaf rcu_node
> > > structure's lock, with an smp_mb__after_unlock_lock(), which will
> > > one way or another be a full memory barrier.  So the reorderings
> > > cannot happen.
> > > 
> > > Unless I am missing something subtle.  ;-)
> > > 
> > 
> > Well, smp_mb__after_unlock_lock() in ARM64 is a no-op, and ARM64's lock
> > doesn't provide a smp_mb().
> > 
> > So my point is more like: synchronize_sched() happens to be a
> > sys_membarrier() because of some implementation detail, and if some day
> > we come up with a much cheaper way to implement sched flavor
> > RCU(hopefully!), synchronize_sched() may be not good for the job. So at
> > least, we'd better document this somewhere?
> 
> Last I heard, ARM's unlock/lock acted as a full barrier.  Will?

Yeah, should do. unlock is release, lock is acquire and we're RCsc.

Will