All of lore.kernel.org
 help / color / mirror / Atom feed
* [BUG] deadlock on rename_lock
@ 2014-02-28  0:45 Jaegeuk Kim
  2014-02-28  4:06 ` Waiman Long
  0 siblings, 1 reply; 2+ messages in thread
From: Jaegeuk Kim @ 2014-02-28  0:45 UTC (permalink / raw)
  To: Al Viro, Waiman Long; +Cc: Linux Kernel, Mailing List

Hi Al,

In the following configuration, I met a deadlock condition like below.

Kernel: 3.14-rc3
Workload: fsstress with 10 threads
Reproducible scenario: N/A

Is it related to this patch?
commit 1370e97bb2eb1ef2df7355204e5a4ba13e12b861
Author: Waiman Long <Waiman.Long@hp.com>
Date:   Thu Sep 12 10:55:34 2013 -0400

    seqlock: Add a new locking reader type 

In d_walk(),
                /*   
                 * might go back up the wrong parent if we have had a
rename
                 * or deletion
                 */
                if (this_parent != child->d_parent ||
                         (child->d_flags & DCACHE_DENTRY_KILLED) ||

--> I suspect that the upper conditions can trigger rename_retry even
though rename_retry was done once before.

                         need_seqretry(&rename_lock, seq)) {
                        spin_unlock(&this_parent->d_lock);
                        rcu_read_unlock();
                        goto rename_retry;
                }    

Thanks,

=============================================
[ INFO: possible recursive locking detected ]
3.14.0-rc3.f2fs+ #10 Tainted: G           O
---------------------------------------------
fsstress/10709 is trying to acquire lock:
 (rename_lock){+.+...}, at: [<ffffffff811cdfbd>] d_walk+0x4d/0x3c0
 
but task is already holding lock:
 (rename_lock){+.+...}, at: [<ffffffff811cdfbd>] d_walk+0x4d/0x3c0

other info that might help us debug this:
 Possible unsafe locking scenario:

       CPU0
       ----
  lock(rename_lock);
  lock(rename_lock);

 *** DEADLOCK ***

 May be due to missing lock nesting notation

4 locks held by fsstress/10709:
 #0:  (sb_writers#11){.+.+.+}, at: [<ffffffff811d84d4>] mnt_want_write
+0x24/0x50
 #1:  (&type->i_mutex_dir_key#3/1){+.+.+.}, at: [<ffffffff811c4c44>]
do_rmdir+0x134/0x1f0
 #2:  (&type->i_mutex_dir_key#3){+.+.+.}, at: [<ffffffff811c4a47>]
vfs_rmdir+0x57/0x120
 #3:  (rename_lock){+.+...}, at: [<ffffffff811cdfbd>] d_walk+0x4d/0x3c0

stack backtrace:
CPU: 3 PID: 10709 Comm: fsstress Tainted: G           O 3.14.0-rc3.f2fs+
#10
Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77
Extreme4, BIOS P2.30 09/21/2012
 ffffffff82173620 ffff88002b1f1bc8 ffffffff81719b27 0000000000000007
 ffffffff82173620 ffff88002b1f1cc8 ffffffff810a72f5 ffff88002b1f1bf8
 0000000000021810 ffff880036cb8000 0000000000000000 ffff88002b1f1d48
Call Trace:
 [<ffffffff81719b27>] dump_stack+0x4e/0x68
 [<ffffffff810a72f5>] __lock_acquire+0x715/0x1e20
 [<ffffffff810c59d8>] ? rcu_irq_exit+0x68/0xb0  
 [<ffffffff81726ddc>] ? retint_restore_args+0xe/0xe
 [<ffffffff810a905e>] lock_acquire+0x8e/0x110
 [<ffffffff811cdfbd>] ? d_walk+0x4d/0x3c0
 [<ffffffff811ce230>] ? d_walk+0x2c0/0x3c0
 [<ffffffff811cd190>] ? umount_collect+0x130/0x130
 [<ffffffff81725d7e>] _raw_spin_lock+0x3e/0x80  
 [<ffffffff811cdfbd>] ? d_walk+0x4d/0x3c0
 [<ffffffff811cdfbd>] d_walk+0x4d/0x3c0
 [<ffffffff811ce081>] ? d_walk+0x111/0x3c0
 [<ffffffff811cd190>] ? umount_collect+0x130/0x130
 [<ffffffff811ce8a8>] shrink_dcache_parent+0x68/0x80
 [<ffffffff811c4a97>] vfs_rmdir+0xa7/0x120
 [<ffffffff813365c9>] ? apparmor_path_rmdir+0x19/0x20 
 [<ffffffff811c4cd3>] do_rmdir+0x1c3/0x1f0
 [<ffffffff81388bbe>] ? trace_hardirqs_on_thunk+0x3a/0x3f
 [<ffffffff811c80e6>] SyS_rmdir+0x16/0x20
 [<ffffffff8172fb12>] system_call_fastpath+0x16/0x1b
BUG: soft lockup - CPU#0 stuck for 22s! [fsstress:10715]
Modules linked in: f2fs(O) snd_hda_codec_hdmi snd_hda_codec_realtek
snd_hda_codec_generic snd_hda_intel snd_hda_codec snd_hwdep i915 snd_pcm
snd_timer drm_kms_helper psmouse drm snd serio_raw soundcore
i2c_algo_bit lpc_ich mac_hid mxm_wmi tpm_tis video wmi lp parport tg3
ptp pps_core [last unloaded: f2fs]
irq event stamp: 11620621
hardirqs last  enabled at (11620621): [<ffffffff810a557d>]
debug_check_no_locks_freed+0x9d/0x170
hardirqs last disabled at (11620620): [<ffffffff810a5520>]
debug_check_no_locks_freed+0x40/0x170
softirqs last  enabled at (11617658): [<ffffffff810575e3>] __do_softirq
+0x1d3/0x2f0
softirqs last disabled at (11617625): [<ffffffff810579d5>] irq_exit
+0xb5/0xc0
CPU: 0 PID: 10715 Comm: fsstress Tainted: G           O 3.14.0-rc3.f2fs+
#10
Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77
Extreme4, BIOS P2.30 09/21/2012
task: ffff880036fcc840 ti: ffff88000ecbe000 task.ti: ffff88000ecbe000
RIP: 0010:[<ffffffff8138771f>]  [<ffffffff8138771f>] delay_tsc+0x4f/0x80
RSP: 0018:ffff88000ecbfc88  EFLAGS: 00000202
RAX: 00000000be2c718a RBX: ffffffff81726ddc RCX: 000000000000001f
RDX: 0000000000035c36 RSI: 00000000be2c716b RDI: 0000000000000001
RBP: ffff88000ecbfc88 R08: 0000000000000000 R09: 0000000000000000
R10: 57ffe8b10853be00 R11: 0000000000000000 R12: ffff88000ecbfbf8
R13: ffff88007cfd4040 R14: ffff88000ecbe000 R15: ffff880036fcc840
FS:  00007f0bdf19d700(0000) GS:ffff88007ce00000(0000)
knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 00007f5f111f09f1 CR3: 00000000188f0000 CR4: 00000000001407f0
Stack:
 ffff88000ecbfc98 ffffffff8138760f ffff88000ecbfcc8 ffffffff810acd64
 ffffffff81c045d0 ffffffff81c045b8 ffffffff81c045d0 ffff88003ee0dfd0
 ffff88000ecbfcf8 ffffffff81725d9e ffffffff811d0112 0000000000000000
Call Trace:
 [<ffffffff8138760f>] __delay+0xf/0x20
 [<ffffffff810acd64>] do_raw_spin_lock+0xf4/0x160
 [<ffffffff81725d9e>] _raw_spin_lock+0x5e/0x80
 [<ffffffff811d0112>] ? d_move+0x22/0x90
 [<ffffffff811d0112>] d_move+0x22/0x90
 [<ffffffff811c52ef>] vfs_rename+0x5ef/0x600
 [<ffffffff811c05bd>] ? lookup_real+0x1d/0x60
 [<ffffffff811c566d>] SYSC_renameat+0x36d/0x3c0
 [<ffffffff811d7f76>] ? mntput_no_expire+0x56/0x190
 [<ffffffff811d7f8e>] ? mntput_no_expire+0x6e/0x190
 [<ffffffff811d7f37>] ? mntput_no_expire+0x17/0x190
 [<ffffffff811d80d4>] ? mntput+0x24/0x40
 [<ffffffff81388bbe>] ? trace_hardirqs_on_thunk+0x3a/0x3f
 [<ffffffff811c889b>] SyS_rename+0x1b/0x20
 [<ffffffff8172fb12>] system_call_fastpath+0x16/0x1b

	
-- 
Jaegeuk Kim
Samsung


^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [BUG] deadlock on rename_lock
  2014-02-28  0:45 [BUG] deadlock on rename_lock Jaegeuk Kim
@ 2014-02-28  4:06 ` Waiman Long
  0 siblings, 0 replies; 2+ messages in thread
From: Waiman Long @ 2014-02-28  4:06 UTC (permalink / raw)
  To: jaegeuk.kim; +Cc: Al Viro, Linux Kernel, Mailing List

On 02/27/2014 07:45 PM, Jaegeuk Kim wrote:
> Hi Al,
>
> In the following configuration, I met a deadlock condition like below.
>
> Kernel: 3.14-rc3
> Workload: fsstress with 10 threads
> Reproducible scenario: N/A
>
> Is it related to this patch?
> commit 1370e97bb2eb1ef2df7355204e5a4ba13e12b861
> Author: Waiman Long<Waiman.Long@hp.com>
> Date:   Thu Sep 12 10:55:34 2013 -0400
>
>      seqlock: Add a new locking reader type
>
> In d_walk(),
>                  /*
>                   * might go back up the wrong parent if we have had a
> rename
>                   * or deletion
>                   */
>                  if (this_parent != child->d_parent ||
>                           (child->d_flags&  DCACHE_DENTRY_KILLED) ||
>
> -->  I suspect that the upper conditions can trigger rename_retry even
> though rename_retry was done once before.
>
>                           need_seqretry(&rename_lock, seq)) {
>                          spin_unlock(&this_parent->d_lock);
>                          rcu_read_unlock();
>                          goto rename_retry;
>                  }
>
> Thanks,
>
>

It seems like the rename_lock may not be able to fully protect against 
the setting of the DCACHE_DENTRY_KILLED flag. Al, should this case be 
handled separately? I am 100% sure if we could just release the lock and 
let it try again without causing infinite loop.

-Longman

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2014-02-28  4:07 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-02-28  0:45 [BUG] deadlock on rename_lock Jaegeuk Kim
2014-02-28  4:06 ` Waiman Long

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.