From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
	id S1752897AbbBRWn1 (ORCPT <rfc822;w@1wt.eu>);
	Wed, 18 Feb 2015 17:43:27 -0500
Received: from bombadil.infradead.org ([198.137.202.9]:47525 "EHLO
	bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org
	with ESMTP id S1752396AbbBRWn0 (ORCPT
	<rfc822;linux-kernel@vger.kernel.org>);
	Wed, 18 Feb 2015 17:43:26 -0500
Date: Wed, 18 Feb 2015 23:43:17 +0100
From: Peter Zijlstra <peterz@infradead.org>
To: Manfred Spraul <manfred@colorfullife.com>
Cc: Oleg Nesterov <oleg@redhat.com>,
        "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
        Kirill Tkhai <ktkhai@parallels.com>, linux-kernel@vger.kernel.org,
        Ingo Molnar <mingo@redhat.com>, Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH 2/2] [PATCH] sched: Add smp_rmb() in task rq locking
 cycles
Message-ID: <20150218224317.GC5029@twins.programming.kicks-ass.net>
References: <20150217104516.12144.85911.stgit@tkhai>
 <1424170021.5749.22.camel@tkhai>
 <20150217121258.GM5029@twins.programming.kicks-ass.net>
 <20150217130523.GV24151@twins.programming.kicks-ass.net>
 <20150217160532.GW4166@linux.vnet.ibm.com>
 <20150217183636.GR5029@twins.programming.kicks-ass.net>
 <20150217215231.GK4166@linux.vnet.ibm.com>
 <20150218155904.GA27687@redhat.com>
 <54E4E479.4050003@colorfullife.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <54E4E479.4050003@colorfullife.com>
User-Agent: Mutt/1.5.21 (2012-12-30)
Sender: linux-kernel-owner@vger.kernel.org
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Feb 18, 2015 at 08:14:01PM +0100, Manfred Spraul wrote:

> >spinlock_t local, global;
> >bool force_global;
> >bool my_lock(bool try_local)
> >{
> >	if (try_local) {
> >		spin_lock(&local);
> >		if (!spin_is_locked(&global)) {
> >			if (!force_global) {
> >				return true;
> >			}
> >		}
> >		spin_unlock(&local);
> >
> >
> >		spin_lock(&global);
> >		spin_unlock_wait(&local);
> >		return false;
> >	}
> >
> >	void my_unlock(bool drop_local)
> >	{
> >		if (drop_local)
> >			spin_unlock(&local);
> >		else
> >			spin_unlock(&global);
> >	}
> >}

> >Another question is do we need a barrier AFTER spin_unlock_wait(). I do not
> >know what ipc/sem.c actually needs, but in general (I think) this does need
> >mb(). Otherwise my_lock / my_unlock itself does not have the proper acq/rel
> >semantics. For example, my_lock(false) can miss the changes which were done
> >under my_lock(true).

> How could that happen?
> I thought that
> thread A:
> 	protected_var = 1234;
> 	spin_unlock(&lock_a)
> 	
> thread B:
> 	spin_lock(&lock_b)
> 	if (protected_var)

> is safe. i.e, there is no need that acquire and releases is done on the same pointer.

Well, just those four statements can of course be executed like:

	CPU0		CPU1

			spin_lock(&b)
			if (prot_var)

	prot_var = 1;
	spin_unlock(&a);

And you would see the old var. Lock a and b are completely independent
here.

Now of course the local/global thing in sysvsem is more complex.

As to what Oleg meant:

X := 0

	CPU0				CPU1

	spin_lock(&global);
					spin_lock(&local);
					X = 1;
					spin_unlock(&local);
	spin_unlock_wait(&local);

	assert(X == 1); /* BOOM */

that assert can trigger, because spin_unlock_wait() are reads, the read
of X can be lifted over and above, before the assignment of X on CPU1.

Again, the sysvsem code is slightly more complex, but I think Oleg is
right, there is no guarantee you'll observe the full critical section of
sem->lock if sem_lock() takes the slow path and does sem_wait_array(),
because of the above.