* 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected
@ 2014-02-12 8:11 Christian Kujau
2014-02-12 8:22 ` Christian Kujau
2014-02-13 19:56 ` Christian Kujau
0 siblings, 2 replies; 6+ messages in thread
From: Christian Kujau @ 2014-02-12 8:11 UTC (permalink / raw)
To: xfs
After upgrading from 3.13-rc8 to 3.14-rc2 on this PowerPC G4 machine, this
happened:
======================================================
[ INFO: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected ]
3.14.0-rc2 #1 Not tainted
------------------------------------------------------
rm/9206 [HC0[0]:SC0[0]:HE1:SE1] is trying to acquire:
(&mm->mmap_sem){++++++}, at: [<c00b3654>] might_fault+0x58/0xa0
and this task is already holding:
(&(&ip->i_lock)->mr_lock){++++-.}, at: [<c020d3ec>]
xfs_ilock_data_map_shared+0x28/0x74
which would create a new lock dependency:
(&(&ip->i_lock)->mr_lock){++++-.} -> (&mm->mmap_sem){++++++}
but this new dependency connects a RECLAIM_FS-irq-safe lock:
(&(&ip->i_lock)->mr_lock){++++-.}
... which became RECLAIM_FS-irq-safe at:
[<c0066b38>] lock_acquire+0x4c/0x68
[<c00615fc>] down_write_nested+0x50/0x9c
[<c01ccf50>] xfs_reclaim_inode+0x108/0x31c
[<c01cd318>] xfs_reclaim_inodes_ag+0x1b4/0x35c
[<c01cde40>] xfs_reclaim_inodes_nr+0x38/0x4c
[<c00d4aec>] super_cache_scan+0x148/0x150
[<c00a4c08>] shrink_slab_node+0x134/0x224
[<c00a52fc>] shrink_slab+0x124/0x13c
[<c00a7900>] kswapd+0x460/0x77c
[<c004f8fc>] kthread+0xbc/0xd0
[<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
to a RECLAIM_FS-irq-unsafe lock:
(&mm->mmap_sem){++++++}
... which became RECLAIM_FS-irq-unsafe at:
... [<c00679fc>] lockdep_trace_alloc+0x84/0x104
[<c009d86c>] __alloc_pages_nodemask+0x88/0x6b4
[<c00161fc>] pte_alloc_one+0x30/0x90
[<c00b3b6c>] __pte_alloc+0x20/0xf4
[<c00bd1d4>] move_page_tables+0x2a0/0x2c4
[<c00d7ff8>] setup_arg_pages+0x20c/0x2c8
[<c0122804>] load_elf_binary+0x378/0x1234
[<c00d73a0>] search_binary_handler+0x98/0x1c8
[<c00d8aa4>] do_execve+0x484/0x574
[<c000425c>] try_to_run_init_process+0x18/0x58
[<c0004a48>] kernel_init+0xac/0x104
[<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
[...]
Full dmesg & .config: http://nerdbynature.de/bits/3.14-rc2/
Thanks,
Christian.
--
BOFH excuse #17:
fat electrons in the lines
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected
2014-02-12 8:11 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected Christian Kujau
@ 2014-02-12 8:22 ` Christian Kujau
2014-02-12 11:05 ` Hugh Dickins
2014-02-13 19:56 ` Christian Kujau
1 sibling, 1 reply; 6+ messages in thread
From: Christian Kujau @ 2014-02-12 8:22 UTC (permalink / raw)
To: xfs; +Cc: Hugh Dickins
On Wed, 12 Feb 2014 at 00:11, Christian Kujau wrote:
> After upgrading from 3.13-rc8 to 3.14-rc2 on this PowerPC G4 machine, this
> happened:
>
> ======================================================
> [ INFO: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected ]
> 3.14.0-rc2 #1 Not tainted
> ------------------------------------------------------
Would this be related to the "mmotm 2014-02-05 list_lru_add lockdep splat"
Hugh posted[0] a few days ago?
Christian.
[0] https://lkml.org/lkml/2014/2/5/878
> rm/9206 [HC0[0]:SC0[0]:HE1:SE1] is trying to acquire:
> (&mm->mmap_sem){++++++}, at: [<c00b3654>] might_fault+0x58/0xa0
>
> and this task is already holding:
> (&(&ip->i_lock)->mr_lock){++++-.}, at: [<c020d3ec>]
> xfs_ilock_data_map_shared+0x28/0x74
> which would create a new lock dependency:
> (&(&ip->i_lock)->mr_lock){++++-.} -> (&mm->mmap_sem){++++++}
>
> but this new dependency connects a RECLAIM_FS-irq-safe lock:
> (&(&ip->i_lock)->mr_lock){++++-.}
> ... which became RECLAIM_FS-irq-safe at:
> [<c0066b38>] lock_acquire+0x4c/0x68
> [<c00615fc>] down_write_nested+0x50/0x9c
> [<c01ccf50>] xfs_reclaim_inode+0x108/0x31c
> [<c01cd318>] xfs_reclaim_inodes_ag+0x1b4/0x35c
> [<c01cde40>] xfs_reclaim_inodes_nr+0x38/0x4c
> [<c00d4aec>] super_cache_scan+0x148/0x150
> [<c00a4c08>] shrink_slab_node+0x134/0x224
> [<c00a52fc>] shrink_slab+0x124/0x13c
> [<c00a7900>] kswapd+0x460/0x77c
> [<c004f8fc>] kthread+0xbc/0xd0
> [<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
>
> to a RECLAIM_FS-irq-unsafe lock:
> (&mm->mmap_sem){++++++}
> ... which became RECLAIM_FS-irq-unsafe at:
> ... [<c00679fc>] lockdep_trace_alloc+0x84/0x104
> [<c009d86c>] __alloc_pages_nodemask+0x88/0x6b4
> [<c00161fc>] pte_alloc_one+0x30/0x90
> [<c00b3b6c>] __pte_alloc+0x20/0xf4
> [<c00bd1d4>] move_page_tables+0x2a0/0x2c4
> [<c00d7ff8>] setup_arg_pages+0x20c/0x2c8
> [<c0122804>] load_elf_binary+0x378/0x1234
> [<c00d73a0>] search_binary_handler+0x98/0x1c8
> [<c00d8aa4>] do_execve+0x484/0x574
> [<c000425c>] try_to_run_init_process+0x18/0x58
> [<c0004a48>] kernel_init+0xac/0x104
> [<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
> [...]
>
> Full dmesg & .config: http://nerdbynature.de/bits/3.14-rc2/
>
> Thanks,
> Christian.
> --
> BOFH excuse #17:
>
> fat electrons in the lines
--
BOFH excuse #17:
fat electrons in the lines
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected
2014-02-12 8:22 ` Christian Kujau
@ 2014-02-12 11:05 ` Hugh Dickins
0 siblings, 0 replies; 6+ messages in thread
From: Hugh Dickins @ 2014-02-12 11:05 UTC (permalink / raw)
To: Christian Kujau; +Cc: xfs
On Wed, 12 Feb 2014, Christian Kujau wrote:
> On Wed, 12 Feb 2014 at 00:11, Christian Kujau wrote:
> > After upgrading from 3.13-rc8 to 3.14-rc2 on this PowerPC G4 machine, this
> > happened:
> >
> > ======================================================
> > [ INFO: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected ]
> > 3.14.0-rc2 #1 Not tainted
> > ------------------------------------------------------
>
> Would this be related to the "mmotm 2014-02-05 list_lru_add lockdep splat"
> Hugh posted[0] a few days ago?
No, that was due to a patch in mmotm or linux-next: it's not in 3.14-rc2.
Hugh
>
> Christian.
>
> [0] https://lkml.org/lkml/2014/2/5/878
>
> > rm/9206 [HC0[0]:SC0[0]:HE1:SE1] is trying to acquire:
> > (&mm->mmap_sem){++++++}, at: [<c00b3654>] might_fault+0x58/0xa0
> >
> > and this task is already holding:
> > (&(&ip->i_lock)->mr_lock){++++-.}, at: [<c020d3ec>]
> > xfs_ilock_data_map_shared+0x28/0x74
> > which would create a new lock dependency:
> > (&(&ip->i_lock)->mr_lock){++++-.} -> (&mm->mmap_sem){++++++}
> >
> > but this new dependency connects a RECLAIM_FS-irq-safe lock:
> > (&(&ip->i_lock)->mr_lock){++++-.}
> > ... which became RECLAIM_FS-irq-safe at:
> > [<c0066b38>] lock_acquire+0x4c/0x68
> > [<c00615fc>] down_write_nested+0x50/0x9c
> > [<c01ccf50>] xfs_reclaim_inode+0x108/0x31c
> > [<c01cd318>] xfs_reclaim_inodes_ag+0x1b4/0x35c
> > [<c01cde40>] xfs_reclaim_inodes_nr+0x38/0x4c
> > [<c00d4aec>] super_cache_scan+0x148/0x150
> > [<c00a4c08>] shrink_slab_node+0x134/0x224
> > [<c00a52fc>] shrink_slab+0x124/0x13c
> > [<c00a7900>] kswapd+0x460/0x77c
> > [<c004f8fc>] kthread+0xbc/0xd0
> > [<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
> >
> > to a RECLAIM_FS-irq-unsafe lock:
> > (&mm->mmap_sem){++++++}
> > ... which became RECLAIM_FS-irq-unsafe at:
> > ... [<c00679fc>] lockdep_trace_alloc+0x84/0x104
> > [<c009d86c>] __alloc_pages_nodemask+0x88/0x6b4
> > [<c00161fc>] pte_alloc_one+0x30/0x90
> > [<c00b3b6c>] __pte_alloc+0x20/0xf4
> > [<c00bd1d4>] move_page_tables+0x2a0/0x2c4
> > [<c00d7ff8>] setup_arg_pages+0x20c/0x2c8
> > [<c0122804>] load_elf_binary+0x378/0x1234
> > [<c00d73a0>] search_binary_handler+0x98/0x1c8
> > [<c00d8aa4>] do_execve+0x484/0x574
> > [<c000425c>] try_to_run_init_process+0x18/0x58
> > [<c0004a48>] kernel_init+0xac/0x104
> > [<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
> > [...]
> >
> > Full dmesg & .config: http://nerdbynature.de/bits/3.14-rc2/
> >
> > Thanks,
> > Christian.
> > --
> > BOFH excuse #17:
> >
> > fat electrons in the lines
> --
> BOFH excuse #17:
>
> fat electrons in the lines
>
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected
2014-02-12 8:11 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected Christian Kujau
2014-02-12 8:22 ` Christian Kujau
@ 2014-02-13 19:56 ` Christian Kujau
2014-02-13 20:42 ` Chris Murphy
2014-02-13 22:11 ` Dave Chinner
1 sibling, 2 replies; 6+ messages in thread
From: Christian Kujau @ 2014-02-13 19:56 UTC (permalink / raw)
To: xfs
On Wed, 12 Feb 2014 at 00:11, Christian Kujau wrote:
> After upgrading from 3.13-rc8 to 3.14-rc2 on this PowerPC G4 machine, this
> happened:
Yesterday a slighly different lockdep warning was printed:
=========================================================
[ INFO: possible irq lock inversion dependency detected ]
3.14.0-rc2 #1 Tainted: G W
---------------------------------------------------------
kswapd0/279 just changed the state of lock:
(&(&ip->i_lock)->mr_lock){++++.?}, at: [<c01c213c>]
xfs_free_eofblocks+0xe8/0x2e0
but this lock took another, RECLAIM_FS-unsafe lock in the past:
(&mm->mmap_sem){++++++}
and interrupts could create inverse lock ordering between them.
other info that might help us debug this:
Possible interrupt unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&mm->mmap_sem);
local_irq_disable();
lock(&(&ip->i_lock)->mr_lock);
lock(&mm->mmap_sem);
<Interrupt>
lock(&(&ip->i_lock)->mr_lock);
*** DEADLOCK ***
2 locks held by kswapd0/279:
#0: (shrinker_rwsem){++++..}, at: [<c00a5218>] shrink_slab+0x40/0x13c
#1: (&type->s_umount_key#36){+++++.}, at: [<c00d4840>] grab_super_passive+0x54/0xc0
the shortest dependencies between 2nd lock and 1st lock:
-> (&mm->mmap_sem){++++++} ops: 58148911 {
HARDIRQ-ON-W at:
[<c0066b38>] lock_acquire+0x4c/0x68
[<c0558724>] down_write+0x4c/0x98
[<c00d88fc>] do_execve+0x2dc/0x574
[<c000425c>] try_to_run_init_process+0x18/0x58
[<c0004a48>] kernel_init+0xac/0x104
[<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
...and on it goes, full dmesg: http://nerdbynature.de/bits/3.14-rc2/mm/
Thanks,
Christian.
--
BOFH excuse #65:
system needs to be rebooted
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected
2014-02-13 19:56 ` Christian Kujau
@ 2014-02-13 20:42 ` Chris Murphy
2014-02-13 22:11 ` Dave Chinner
1 sibling, 0 replies; 6+ messages in thread
From: Chris Murphy @ 2014-02-13 20:42 UTC (permalink / raw)
To: Christian Kujau; +Cc: xfs
On Feb 13, 2014, at 12:56 PM, Christian Kujau <lists@nerdbynature.de> wrote:
> On Wed, 12 Feb 2014 at 00:11, Christian Kujau wrote:
>> After upgrading from 3.13-rc8 to 3.14-rc2 on this PowerPC G4 machine, this
>> happened:
>
> Yesterday a slighly different lockdep warning was printed:
>
> =========================================================
> [ INFO: possible irq lock inversion dependency detected ]
> 3.14.0-rc2 #1 Tainted: G W
> ---------------------------------------------------------
> kswapd0/279 just changed the state of lock:
> (&(&ip->i_lock)->mr_lock){++++.?}, at: [<c01c213c>]
> xfs_free_eofblocks+0xe8/0x2e0
> but this lock took another, RECLAIM_FS-unsafe lock in the past:
> (&mm->mmap_sem){++++++}
>
> and interrupts could create inverse lock ordering between them.
>
>
> other info that might help us debug this:
> Possible interrupt unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(&mm->mmap_sem);
> local_irq_disable();
> lock(&(&ip->i_lock)->mr_lock);
> lock(&mm->mmap_sem);
> <Interrupt>
> lock(&(&ip->i_lock)->mr_lock);
>
> *** DEADLOCK ***
>
> 2 locks held by kswapd0/279:
> #0: (shrinker_rwsem){++++..}, at: [<c00a5218>] shrink_slab+0x40/0x13c
> #1: (&type->s_umount_key#36){+++++.}, at: [<c00d4840>] grab_super_passive+0x54/0xc0
> the shortest dependencies between 2nd lock and 1st lock:
> -> (&mm->mmap_sem){++++++} ops: 58148911 {
> HARDIRQ-ON-W at:
> [<c0066b38>] lock_acquire+0x4c/0x68
> [<c0558724>] down_write+0x4c/0x98
> [<c00d88fc>] do_execve+0x2dc/0x574
> [<c000425c>] try_to_run_init_process+0x18/0x58
> [<c0004a48>] kernel_init+0xac/0x104
> [<c0010ae4>] ret_from_kernel_thread+0x5c/0x64
>
>
> …and on it goes, full dmesg: http://nerdbynature.de/bits/3.14-rc2/mm/
A user is seeing something similar on Fedora 3.14.x kernels with Btrfs also.
https://bugzilla.redhat.com/show_bug.cgi?id=1062439
Chris Murphy
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected
2014-02-13 19:56 ` Christian Kujau
2014-02-13 20:42 ` Chris Murphy
@ 2014-02-13 22:11 ` Dave Chinner
1 sibling, 0 replies; 6+ messages in thread
From: Dave Chinner @ 2014-02-13 22:11 UTC (permalink / raw)
To: Christian Kujau; +Cc: xfs
On Thu, Feb 13, 2014 at 11:56:36AM -0800, Christian Kujau wrote:
> On Wed, 12 Feb 2014 at 00:11, Christian Kujau wrote:
> > After upgrading from 3.13-rc8 to 3.14-rc2 on this PowerPC G4 machine, this
> > happened:
>
> Yesterday a slighly different lockdep warning was printed:
>
> =========================================================
> [ INFO: possible irq lock inversion dependency detected ]
> 3.14.0-rc2 #1 Tainted: G W
> ---------------------------------------------------------
> kswapd0/279 just changed the state of lock:
> (&(&ip->i_lock)->mr_lock){++++.?}, at: [<c01c213c>]
> xfs_free_eofblocks+0xe8/0x2e0
> but this lock took another, RECLAIM_FS-unsafe lock in the past:
> (&mm->mmap_sem){++++++}
>
> and interrupts could create inverse lock ordering between them.
False positive - it's confusing directory locking orders with
regular files. Already reported, but the devs responsible for the
lockdep regression have not responded. Looks like I'll have to
fix it myself. I'll see what I can do today.
Cheers,
Dave.
--
Dave Chinner
david@fromorbit.com
_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2014-02-13 22:12 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-12 8:11 3.14-rc2: RECLAIM_FS-safe -> RECLAIM_FS-unsafe lock order detected Christian Kujau
2014-02-12 8:22 ` Christian Kujau
2014-02-12 11:05 ` Hugh Dickins
2014-02-13 19:56 ` Christian Kujau
2014-02-13 20:42 ` Chris Murphy
2014-02-13 22:11 ` Dave Chinner
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.