LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Peter Zijlstra <a.p.zijlstra@chello.nl>
To: Roland Dreier <roland@kernel.org>
Cc: Ingo Molnar <mingo@elte.hu>,
	linux-kernel@vger.kernel.org, rostedt <rostedt@goodmis.org>
Subject: Re: lockdep false positive in double_lock_balance()?
Date: Thu, 17 May 2012 21:19:46 +0200
Message-ID: <1337282386.4281.77.camel@twins> (raw)
In-Reply-To: <1337198420-5062-1-git-send-email-roland@kernel.org>

On Wed, 2012-05-16 at 13:00 -0700, Roland Dreier wrote:
> Hi scheduler hackers,
> 
> I'm very occasionally seeing the lockdep warning below on our boxes
> running 2.6.39 (PREEMPT=n, so "unfair" _double_lock_balance()).  I
> think I see the explanation, and it's probably not even worth fixing:
> 
> On the unlock side, we have:

<snip>

> while on the lock side we have:

<snip>

> So it seems we have the following (purely lockdep-related) race:
> 
>     unlock:				lock:
> 
> 					if (unlikely(!raw_spin_trylock(&busiest->lock))) { //fail to lock
> 
>     raw_spin_unlock(&busiest->lock);
> 
> 						if (busiest < this_rq) { //not true
> 						} else
> 							raw_spin_lock_nested(&busiest->lock,
> 									      SINGLE_DEPTH_NESTING);
> 
>     lock_set_subclass(&this_rq->lock.dep_map, 0, _RET_IP_); //too late
> 
> where we end up trying to take a second lock with SINGLE_DEPTH_NESTING
> before we've promoted our first lock to subclass 0.

*phew* you actually made me think there ;-)

Anyway, it all sounds very plausible, which is what threw me, but its
wrong :-)

The race you describe exists, except that's not how lockdep works. Both
cpu's would have a different task (one would hope to presume) and the
held lock stack is per task. So even if busiest_rq on cpu1 (lock case)
is the same lock as this_rq on cpu0 (unlock case), they're in different
stacks with different states.

> So does this make sense?

Almost :-)

> Here's the actual lockdep warning:

> [89945.640512]  [<ffffffff8103fa1a>] double_lock_balance+0x5a/0x90
> [89945.640568]  [<ffffffff8104c546>] push_rt_task+0xc6/0x290

this is the clue.. if you look at that code you'll find the
double_lock_balance() in question is the one in find_lock_lowest_rq()
[yay for inlining].

Now find_lock_lowest_rq() has a bug.. it fails to use
double_unlock_balance() in one exit path, if this results in a retry in
push_rt_task() we'll call double_lock_balance() again, at which point
we'll run into said issue.

Presumably this is all rather rare..

Something like this should fix it I think..


---
 kernel/sched/rt.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/sched/rt.c b/kernel/sched/rt.c
index c5565c3..b649108 100644
--- a/kernel/sched/rt.c
+++ b/kernel/sched/rt.c
@@ -1556,7 +1556,7 @@ static struct rq *find_lock_lowest_rq(struct task_struct *task, struct rq *rq)
 				     task_running(rq, task) ||
 				     !task->on_rq)) {
 
-				raw_spin_unlock(&lowest_rq->lock);
+				double_unlock_balance(rq, lowest_rq);
 				lowest_rq = NULL;
 				break;
 			}


  reply index

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <1335314100-532-1-git-send-email-roland@kernel.org>
     [not found] ` <alpine.LSU.2.00.1204251411460.4188@eggly.anvils>
2012-05-16 20:00   ` Roland Dreier
2012-05-17 19:19     ` Peter Zijlstra [this message]
2012-05-18 14:43       ` Steven Rostedt
2012-05-18 16:42       ` Roland Dreier
2012-06-06 15:54       ` [tip:sched/urgent] sched/rt: Fix lockdep annotation within find_lock_lowest_rq() tip-bot for Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1337282386.4281.77.camel@twins \
    --to=a.p.zijlstra@chello.nl \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=roland@kernel.org \
    --cc=rostedt@goodmis.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lore.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lore.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lore.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lore.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lore.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lore.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lore.kernel.org/lkml/7 lkml/git/7.git
	git clone --mirror https://lore.kernel.org/lkml/8 lkml/git/8.git
	git clone --mirror https://lore.kernel.org/lkml/9 lkml/git/9.git
	git clone --mirror https://lore.kernel.org/lkml/10 lkml/git/10.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lore.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git