* 3.0.4 + rt12: deadlock
@ 2011-08-31 17:08 Fernando Lopez-Lezcano
2011-09-01 19:38 ` Thomas Gleixner
0 siblings, 1 reply; 3+ messages in thread
From: Fernando Lopez-Lezcano @ 2011-08-31 17:08 UTC (permalink / raw)
To: linux-rt-users
Cc: LKML, Thomas Gleixner, Paul E. McKenney, efault, Fernando Lopez-Lezcano
[-- Attachment #1: Type: text/plain, Size: 2639 bytes --]
Booting 3.0.4 + rt12 on a quadcore workstation (running fc14) gave me this:
----
=============================================
[ INFO: possible recursive locking detected ]
3.0.4-1.rt12.1.fc14.ccrma.i686.rtPAE #1
---------------------------------------------
swapper/0 is trying to acquire lock:
(&parent->list_lock){+.+...}, at: [<c05054ce>]
__cache_free.clone.27+0x45/0xc4
but task is already holding lock:
(&parent->list_lock){+.+...}, at: [<c050662c>] do_tune_cpucache+0xf0/0x2b0
other info that might help us debug this:
Possible unsafe locking scenario:
CPU0
----
lock(&parent->list_lock);
lock(&parent->list_lock);
*** DEADLOCK ***
May be due to missing lock nesting notation
3 locks held by swapper/0:
#0: (cache_chain_mutex){+.+...}, at: [<c0bedb96>]
kmem_cache_init_late+0x15/0x61
#1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [<c0504a53>]
__local_lock_irq+0x1e/0x5b
#2: (&parent->list_lock){+.+...}, at: [<c050662c>]
do_tune_cpucache+0xf0/0x2b0
stack backtrace:
Pid: 0, comm: swapper Not tainted 3.0.4-1.rt12.1.fc14.ccrma.i686.rtPAE #1
Call Trace:
[<c086e6f6>] ? printk+0x2d/0x2f
[<c0479b36>] __lock_acquire+0x8b3/0xc2f
[<c086fd96>] ? rt_spin_lock_slowlock+0x67/0x170
[<c04790ee>] ? mark_lock+0x26/0x1bb
[<c05054ce>] ? __cache_free.clone.27+0x45/0xc4
[<c047a372>] lock_acquire+0xde/0x11d
[<c05054ce>] ? __cache_free.clone.27+0x45/0xc4
[<c08703b4>] rt_spin_lock+0x3d/0x43
[<c05054ce>] ? __cache_free.clone.27+0x45/0xc4
[<c05054ce>] __cache_free.clone.27+0x45/0xc4
[<c0437c83>] ? test_ti_thread_flag+0x8/0x10
[<c0505303>] kmem_cache_free+0x73/0xe1
[<c05053c0>] slab_destroy+0x4f/0x53
[<c0505458>] free_block+0x94/0xc5
[<c0506645>] do_tune_cpucache+0x109/0x2b0
[<c05069c7>] enable_cpucache+0x7b/0xa7
[<c0bedba7>] kmem_cache_init_late+0x26/0x61
[<c0bcd68a>] start_kernel+0x24f/0x367
[<c0bcd1d1>] ? loglevel+0x1a/0x1a
[<c0bcd13b>] ? reserve_ebda_region+0x70/0x72
[<c0bcd0c3>] i386_start_kernel+0xb2/0xba
Console: colour VGA+ 80x25
console [tty0] enabled
Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
... MAX_LOCKDEP_SUBCLASSES: 8
... MAX_LOCK_DEPTH: 48
... MAX_LOCKDEP_KEYS: 8191
... CLASSHASH_SIZE: 4096
... MAX_LOCKDEP_ENTRIES: 32768
... MAX_LOCKDEP_CHAINS: 65536
... CHAINHASH_SIZE: 32768
----
I started working and a little while later the machine froze (jack +
heavy prioritized udp traffic in eth1 - with the r8169 driver). It
recognized alt-sysrq boot so it was not completely dead. Nothing left on
the logs to see.
Full config attached.
-- Fernando
[-- Attachment #2: config.bz2 --]
[-- Type: application/x-bzip, Size: 28822 bytes --]
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: 3.0.4 + rt12: deadlock
2011-08-31 17:08 3.0.4 + rt12: deadlock Fernando Lopez-Lezcano
@ 2011-09-01 19:38 ` Thomas Gleixner
2011-09-01 20:52 ` Peter Zijlstra
0 siblings, 1 reply; 3+ messages in thread
From: Thomas Gleixner @ 2011-09-01 19:38 UTC (permalink / raw)
To: Fernando Lopez-Lezcano
Cc: linux-rt-users, LKML, Paul E. McKenney, efault, Peter Zijlstra, LKML
On Wed, 31 Aug 2011, Fernando Lopez-Lezcano wrote:
> Booting 3.0.4 + rt12 on a quadcore workstation (running fc14) gave me this:
>
> ----
> =============================================
> [ INFO: possible recursive locking detected ]
> 3.0.4-1.rt12.1.fc14.ccrma.i686.rtPAE #1
> ---------------------------------------------
> swapper/0 is trying to acquire lock:
> (&parent->list_lock){+.+...}, at: [<c05054ce>]
> __cache_free.clone.27+0x45/0xc4
>
> but task is already holding lock:
> (&parent->list_lock){+.+...}, at: [<c050662c>] do_tune_cpucache+0xf0/0x2b0
>
> other info that might help us debug this:
> Possible unsafe locking scenario:
>
> CPU0
> ----
> lock(&parent->list_lock);
> lock(&parent->list_lock);
That's something which has to do with debugging options (debugobjects
IIRC). There was some attempt to fix that, but that might have gone
lost in my vacation and the following futile attempt to take care of
the resulting backlog. Peter ???
Thanks,
tglx
> *** DEADLOCK ***
>
> May be due to missing lock nesting notation
>
> 3 locks held by swapper/0:
> #0: (cache_chain_mutex){+.+...}, at: [<c0bedb96>]
> kmem_cache_init_late+0x15/0x61
> #1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [<c0504a53>]
> __local_lock_irq+0x1e/0x5b
> #2: (&parent->list_lock){+.+...}, at: [<c050662c>]
> do_tune_cpucache+0xf0/0x2b0
>
> stack backtrace:
> Pid: 0, comm: swapper Not tainted 3.0.4-1.rt12.1.fc14.ccrma.i686.rtPAE #1
> Call Trace:
> [<c086e6f6>] ? printk+0x2d/0x2f
> [<c0479b36>] __lock_acquire+0x8b3/0xc2f
> [<c086fd96>] ? rt_spin_lock_slowlock+0x67/0x170
> [<c04790ee>] ? mark_lock+0x26/0x1bb
> [<c05054ce>] ? __cache_free.clone.27+0x45/0xc4
> [<c047a372>] lock_acquire+0xde/0x11d
> [<c05054ce>] ? __cache_free.clone.27+0x45/0xc4
> [<c08703b4>] rt_spin_lock+0x3d/0x43
> [<c05054ce>] ? __cache_free.clone.27+0x45/0xc4
> [<c05054ce>] __cache_free.clone.27+0x45/0xc4
> [<c0437c83>] ? test_ti_thread_flag+0x8/0x10
> [<c0505303>] kmem_cache_free+0x73/0xe1
> [<c05053c0>] slab_destroy+0x4f/0x53
> [<c0505458>] free_block+0x94/0xc5
> [<c0506645>] do_tune_cpucache+0x109/0x2b0
> [<c05069c7>] enable_cpucache+0x7b/0xa7
> [<c0bedba7>] kmem_cache_init_late+0x26/0x61
> [<c0bcd68a>] start_kernel+0x24f/0x367
> [<c0bcd1d1>] ? loglevel+0x1a/0x1a
> [<c0bcd13b>] ? reserve_ebda_region+0x70/0x72
> [<c0bcd0c3>] i386_start_kernel+0xb2/0xba
> Console: colour VGA+ 80x25
> console [tty0] enabled
> Lock dependency validator: Copyright (c) 2006 Red Hat, Inc., Ingo Molnar
> ... MAX_LOCKDEP_SUBCLASSES: 8
> ... MAX_LOCK_DEPTH: 48
> ... MAX_LOCKDEP_KEYS: 8191
> ... CLASSHASH_SIZE: 4096
> ... MAX_LOCKDEP_ENTRIES: 32768
> ... MAX_LOCKDEP_CHAINS: 65536
> ... CHAINHASH_SIZE: 32768
> ----
>
> I started working and a little while later the machine froze (jack + heavy
> prioritized udp traffic in eth1 - with the r8169 driver). It recognized
> alt-sysrq boot so it was not completely dead. Nothing left on the logs to see.
>
> Full config attached.
> -- Fernando
>
^ permalink raw reply [flat|nested] 3+ messages in thread* Re: 3.0.4 + rt12: deadlock
2011-09-01 19:38 ` Thomas Gleixner
@ 2011-09-01 20:52 ` Peter Zijlstra
0 siblings, 0 replies; 3+ messages in thread
From: Peter Zijlstra @ 2011-09-01 20:52 UTC (permalink / raw)
To: Thomas Gleixner
Cc: Fernando Lopez-Lezcano, linux-rt-users, LKML, Paul E. McKenney, efault
On Thu, 2011-09-01 at 21:38 +0200, Thomas Gleixner wrote:
> > =============================================
> > [ INFO: possible recursive locking detected ]
> > 3.0.4-1.rt12.1.fc14.ccrma.i686.rtPAE #1
> > ---------------------------------------------
> > swapper/0 is trying to acquire lock:
> > (&parent->list_lock){+.+...}, at: [<c05054ce>]
> > __cache_free.clone.27+0x45/0xc4
> >
> > but task is already holding lock:
> > (&parent->list_lock){+.+...}, at: [<c050662c>] do_tune_cpucache
> +0xf0/0x2b0
> >
> > other info that might help us debug this:
> > Possible unsafe locking scenario:
> >
> > CPU0
> > ----
> > lock(&parent->list_lock);
> > lock(&parent->list_lock);
>
>
>
> > *** DEADLOCK ***
> >
> > May be due to missing lock nesting notation
> >
> > 3 locks held by swapper/0:
> > #0: (cache_chain_mutex){+.+...}, at: [<c0bedb96>]
> > kmem_cache_init_late+0x15/0x61
> > #1: (&per_cpu(slab_lock, __cpu).lock){+.+...}, at: [<c0504a53>]
> > __local_lock_irq+0x1e/0x5b
> > #2: (&parent->list_lock){+.+...}, at: [<c050662c>]
> > do_tune_cpucache+0xf0/0x2b0
> That's something which has to do with debugging options (debugobjects
> IIRC). There was some attempt to fix that, but that might have gone
> lost in my vacation and the following futile attempt to take care of
> the resulting backlog. Peter ???
Looks like the one supposedly cured by:
patches/peter_zijlstra-slab_lockdep-annotate_the_locks_before_using.patch
which should be in -rt12
will have a peek, never reproduced for me though..
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2011-09-01 20:52 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-08-31 17:08 3.0.4 + rt12: deadlock Fernando Lopez-Lezcano
2011-09-01 19:38 ` Thomas Gleixner
2011-09-01 20:52 ` Peter Zijlstra
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.