All of lore.kernel.org
 help / color / mirror / Atom feed
From: Pekka Enberg <penberg@kernel.org>
To: Peter Zijlstra <peterz@infradead.org>
Cc: Fernando Lopez-Lezcano <nando@ccrma.Stanford.EDU>,
	Thomas Gleixner <tglx@linutronix.de>,
	LKML <linux-kernel@vger.kernel.org>,
	linux-rt-users <linux-rt-users@vger.kernel.org>,
	"Paul E. McKenney" <paulmck@linux.vnet.ibm.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Jason Wessel <jason.wessel@windriver.com>,
	cl@linux.com
Subject: Re: [ANNOUNCE] 3.0-rt4
Date: Fri, 29 Jul 2011 01:55:37 +0300	[thread overview]
Message-ID: <1311893737.24862.51.camel@jaguar> (raw)
In-Reply-To: <1311888176.2617.379.camel@laptop>

On Thu, 2011-07-28 at 23:22 +0200, Peter Zijlstra wrote:
> On Thu, 2011-07-28 at 12:16 -0700, Fernando Lopez-Lezcano wrote:
> > [    0.000000] =============================================
> > [    0.000000] [ INFO: possible recursive locking detected ]
> > [    0.000000] 3.0.0-1.rt5.1.fc15.ccrma.i686.rtPAE #1
> > [    0.000000] ---------------------------------------------
> > [    0.000000] swapper/0 is trying to acquire lock:
> > [    0.000000]  (&parent->list_lock){+.+...}, at: [<c04fb406>] __cache_free+0x43/0xc3
> > [    0.000000]
> > [    0.000000] but task is already holding lock:
> > [    0.000000]  (&parent->list_lock){+.+...}, at: [<c04fc538>] do_tune_cpucache+0xf2/0x2bb
> > [    0.000000]
> > [    0.000000] other info that might help us debug this:
> > [    0.000000]  Possible unsafe locking scenario:
> > [    0.000000]
> > [    0.000000]        CPU0
> > [    0.000000]        ----
> > [    0.000000]   lock(&parent->list_lock);
> > [    0.000000]   lock(&parent->list_lock);
> > [    0.000000]
> > [    0.000000]  *** DEADLOCK ***
> > [    0.000000]
> > [    0.000000]  May be due to missing lock nesting notation
> > [    0.000000]
> > [    0.000000] 3 locks held by swapper/0:
> > [    0.000000]  #0:  (cache_chain_mutex){+.+...}, at: [<c0bd9d2b>] kmem_cache_init_late+0xe/0x61
> > [    0.000000]  #1:  (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [<c04faa65>] __local_lock_irq+0x1e/0x5b
> > [    0.000000]  #2:  (&parent->list_lock){+.+...}, at: [<c04fc538>] do_tune_cpucache+0xf2/0x2bb
> > [    0.000000]
> > [    0.000000] stack backtrace:
> > [    0.000000] Pid: 0, comm: swapper Not tainted 3.0.0-1.rt5.1.fc15.ccrma.i686.rtPAE #1
> > [    0.000000] Call Trace:
> > [    0.000000]  [<c0856355>] ? printk+0x2d/0x2f
> > [    0.000000]  [<c0474a4b>] __lock_acquire+0x805/0xb57
> > [    0.000000]  [<c0472604>] ? lock_release_holdtime.part.10+0x4b/0x51
> > [    0.000000]  [<c085ecb4>] ? _raw_spin_unlock+0x31/0x3d
> > [    0.000000]  [<c085dbc5>] ? rt_spin_lock_slowlock+0x75/0x190
> > [    0.000000]  [<c04720c3>] ? trace_hardirqs_off+0xb/0xd
> > [    0.000000]  [<c04fb406>] ? __cache_free+0x43/0xc3
> > [    0.000000]  [<c0475215>] lock_acquire+0xde/0x11d
> > [    0.000000]  [<c04fb406>] ? __cache_free+0x43/0xc3
> > [    0.000000]  [<c085e24f>] rt_spin_lock+0x50/0x56
> > [    0.000000]  [<c04fb406>] ? __cache_free+0x43/0xc3
> > [    0.000000]  [<c04fb406>] __cache_free+0x43/0xc3
> > [    0.000000]  [<c043646d>] ? test_ti_thread_flag+0x8/0x10
> > [    0.000000]  [<c04fb23f>] kmem_cache_free+0x6c/0xdc
> > [    0.000000]  [<c04fb2fe>] slab_destroy+0x4f/0x53
> > [    0.000000]  [<c04fb396>] free_block+0x94/0xc1
> > [    0.000000]  [<c04fc551>] do_tune_cpucache+0x10b/0x2bb
> > [    0.000000]  [<c04fc8dc>] enable_cpucache+0x7b/0xa7
> > [    0.000000]  [<c0bd9d3c>] kmem_cache_init_late+0x1f/0x61
> > [    0.000000]  [<c0bba687>] start_kernel+0x24c/0x363
> > [    0.000000]  [<c0bba1c4>] ? loglevel+0x18/0x18
> > [    0.000000]  [<c0bba0ba>] i386_start_kernel+0xa9/0xaf 
> 
> Ooh, fun.. one does wonder why mainline doesn't show this..
> 
> This is the normal OFF_SLAB recursion, and the reason this shows up is
> because this is ran before we do the lockdep fixup.
> 
> Fernando, does something like the below (not actually against -rt, but
> it shouldn't matter much) fix things?
> 
> ---
> Subject: slab, lockdep: Annotate the locks before using
> 
> Fernando found we hit the regular OFF_SLAB 'recursion' before we
> annotate the locks, cure this.
> 
> Reported-by: Fernando Lopez-Lezcano <nando@ccrma.Stanford.EDU>
> Signed-off-by: Peter Zijlstra <a.p.zijlstra@chello.nl>

If you want to pick this up in the lockdep tree:

Acked-by: Pekka Enberg <penberg@kernel.org>

> ---
> Index: linux-2.6/mm/slab.c
> ===================================================================
> --- linux-2.6.orig/mm/slab.c
> +++ linux-2.6/mm/slab.c
> @@ -1665,6 +1665,9 @@ void __init kmem_cache_init_late(void)
>  {
>  	struct kmem_cache *cachep;
>  
> +	/* Annotate slab for lockdep -- annotate the malloc caches */
> +	init_lock_keys();
> +
>  	/* 6) resize the head arrays to their final sizes */
>  	mutex_lock(&cache_chain_mutex);
>  	list_for_each_entry(cachep, &cache_chain, next)
> @@ -1675,9 +1678,6 @@ void __init kmem_cache_init_late(void)
>  	/* Done! */
>  	g_cpucache_up = FULL;
>  
> -	/* Annotate slab for lockdep -- annotate the malloc caches */
> -	init_lock_keys();
> -
>  	/*
>  	 * Register a cpu startup notifier callback that initializes
>  	 * cpu_cache_get for all new cpus
> 
> 



  reply	other threads:[~2011-07-28 22:55 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-07-27 21:37 [ANNOUNCE] 3.0-rt4 Thomas Gleixner
2011-07-27 21:57 ` Frank Rowand
2011-07-27 22:05   ` Thomas Gleixner
2011-07-27 22:27     ` Frank Rowand
2011-07-28  7:33 ` Nikita V. Youshchenko
2011-07-28  8:06   ` Thomas Gleixner
2011-07-28  8:24     ` Nikita V. Youshchenko
2011-07-28  8:44       ` Peter Zijlstra
2011-07-28  8:51         ` Nikita V. Youshchenko
2011-07-28  9:21           ` Anca Emanuel
2011-07-28 16:10             ` Thomas Gleixner
2011-07-28  9:37           ` Thomas Gleixner
2011-07-28  8:43     ` Peter Zijlstra
2011-07-28  9:41 ` Yong Zhang
2011-07-28  9:41   ` Yong Zhang
2011-07-28  9:49   ` Thomas Gleixner
2011-07-28 11:22 ` N, Mugunthan V
2011-07-28 11:36   ` Thomas Gleixner
2011-07-28 15:59 ` hermann
2011-07-28 17:41 ` Paul E. McKenney
2011-07-28 17:43   ` Thomas Gleixner
2011-07-28 18:32     ` Paul E. McKenney
2011-07-28 19:05       ` Thomas Gleixner
2011-07-28 19:34         ` Paul E. McKenney
2011-07-28 20:29           ` Paul E. McKenney
2011-07-28 17:42 ` Jason Wessel
2011-07-28 17:49   ` Thomas Gleixner
2011-07-28 19:16 ` Fernando Lopez-Lezcano
2011-07-28 19:36   ` Thomas Gleixner
2011-07-29  0:13     ` Fernando Lopez-Lezcano
2011-07-28 21:22   ` Peter Zijlstra
2011-07-28 21:22     ` Peter Zijlstra
2011-07-28 22:55     ` Pekka Enberg [this message]
2011-07-29  0:17     ` Fernando Lopez-Lezcano
2011-08-04  8:36     ` [tip:core/urgent] slab, lockdep: Annotate the locks before using them tip-bot for Peter Zijlstra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1311893737.24862.51.camel@jaguar \
    --to=penberg@kernel.org \
    --cc=cl@linux.com \
    --cc=jason.wessel@windriver.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-users@vger.kernel.org \
    --cc=nando@ccrma.Stanford.EDU \
    --cc=paulmck@linux.vnet.ibm.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.