* [syzbot] possible deadlock in loop_add
@ 2021-07-10 11:01 syzbot
[not found] ` <20210710131638.605-1-hdanton@sina.com>
0 siblings, 1 reply; 5+ messages in thread
From: syzbot @ 2021-07-10 11:01 UTC (permalink / raw)
To: axboe, bp, christian.brauner, corbet, hch, hpa, jmattson,
johannes.thumshirn, joro, josef, kvm, linux-block, linux-doc,
linux-fsdevel, linux-kernel, mingo, pbonzini, seanjc,
syzkaller-bugs, tglx, viro, vkuznets, wanpengli, x86
Hello,
syzbot found the following issue on:
HEAD commit: ee268dee Add linux-next specific files for 20210707
git tree: linux-next
console output: https://syzkaller.appspot.com/x/log.txt?x=12c39ae2300000
kernel config: https://syzkaller.appspot.com/x/.config?x=59e1e3bbc3afca75
dashboard link: https://syzkaller.appspot.com/bug?extid=118992efda475c16dfb0
syz repro: https://syzkaller.appspot.com/x/repro.syz?x=14698794300000
C reproducer: https://syzkaller.appspot.com/x/repro.c?x=13e25ee4300000
The issue was bisected to:
commit 0f00b82e5413571ed225ddbccad6882d7ea60bc7
Author: Christoph Hellwig <hch@lst.de>
Date: Mon Mar 8 07:45:50 2021 +0000
block: remove the revalidate_disk method
bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=14bb406c300000
final oops: https://syzkaller.appspot.com/x/report.txt?x=16bb406c300000
console output: https://syzkaller.appspot.com/x/log.txt?x=12bb406c300000
IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+118992efda475c16dfb0@syzkaller.appspotmail.com
Fixes: 0f00b82e5413 ("block: remove the revalidate_disk method")
======================================================
WARNING: possible circular locking dependency detected
5.13.0-next-20210707-syzkaller #0 Not tainted
------------------------------------------------------
systemd-udevd/8674 is trying to acquire lock:
ffffffff8c4875c8 (loop_ctl_mutex){+.+.}-{3:3}, at: loop_add+0x9c/0x8c0 drivers/block/loop.c:2250
but task is already holding lock:
ffffffff8c1f3e08 (major_names_lock){+.+.}-{3:3}, at: blk_request_module+0x25/0x1d0 block/genhd.c:657
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (major_names_lock){+.+.}-{3:3}:
__mutex_lock_common kernel/locking/mutex.c:959 [inline]
__mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
__register_blkdev+0x2b/0x3e0 block/genhd.c:216
register_mtd_blktrans+0x85/0x3c0 drivers/mtd/mtd_blkdevs.c:531
do_one_initcall+0x103/0x650 init/main.c:1285
do_initcall_level init/main.c:1360 [inline]
do_initcalls init/main.c:1376 [inline]
do_basic_setup init/main.c:1396 [inline]
kernel_init_freeable+0x6b8/0x741 init/main.c:1598
kernel_init+0x1a/0x1d0 init/main.c:1490
ret_from_fork+0x1f/0x30 arch/x86/entry/entry_64.S:295
-> #2 (mtd_table_mutex){+.+.}-{3:3}:
__mutex_lock_common kernel/locking/mutex.c:959 [inline]
__mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
blktrans_open+0x69/0x600 drivers/mtd/mtd_blkdevs.c:210
blkdev_get_whole+0xa1/0x420 fs/block_dev.c:1251
blkdev_get_by_dev.part.0+0x30c/0xdd0 fs/block_dev.c:1415
blkdev_get_by_dev fs/block_dev.c:1504 [inline]
blkdev_open+0x295/0x300 fs/block_dev.c:1510
do_dentry_open+0x4c8/0x11d0 fs/open.c:826
do_open fs/namei.c:3374 [inline]
path_openat+0x1c23/0x27f0 fs/namei.c:3507
do_filp_open+0x1aa/0x400 fs/namei.c:3534
do_sys_openat2+0x16d/0x420 fs/open.c:1204
do_sys_open fs/open.c:1220 [inline]
__do_sys_open fs/open.c:1228 [inline]
__se_sys_open fs/open.c:1224 [inline]
__x64_sys_open+0x119/0x1c0 fs/open.c:1224
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
-> #1 (&disk->open_mutex){+.+.}-{3:3}:
__mutex_lock_common kernel/locking/mutex.c:959 [inline]
__mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
del_gendisk+0x8b/0x770 block/genhd.c:587
loop_remove drivers/block/loop.c:2347 [inline]
loop_control_remove drivers/block/loop.c:2396 [inline]
loop_control_ioctl+0x3b5/0x450 drivers/block/loop.c:2428
vfs_ioctl fs/ioctl.c:51 [inline]
__do_sys_ioctl fs/ioctl.c:1069 [inline]
__se_sys_ioctl fs/ioctl.c:1055 [inline]
__x64_sys_ioctl+0x193/0x200 fs/ioctl.c:1055
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
-> #0 (loop_ctl_mutex){+.+.}-{3:3}:
check_prev_add kernel/locking/lockdep.c:3051 [inline]
check_prevs_add kernel/locking/lockdep.c:3174 [inline]
validate_chain kernel/locking/lockdep.c:3789 [inline]
__lock_acquire+0x2a07/0x54a0 kernel/locking/lockdep.c:5015
lock_acquire kernel/locking/lockdep.c:5625 [inline]
lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5590
__mutex_lock_common kernel/locking/mutex.c:959 [inline]
__mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
loop_add+0x9c/0x8c0 drivers/block/loop.c:2250
loop_probe+0x6a/0x80 drivers/block/loop.c:2360
blk_request_module+0x111/0x1d0 block/genhd.c:660
blkdev_get_no_open+0x1d5/0x250 fs/block_dev.c:1332
blkdev_get_by_dev.part.0+0x25/0xdd0 fs/block_dev.c:1395
blkdev_get_by_dev fs/block_dev.c:1504 [inline]
blkdev_open+0x295/0x300 fs/block_dev.c:1510
do_dentry_open+0x4c8/0x11d0 fs/open.c:826
do_open fs/namei.c:3374 [inline]
path_openat+0x1c23/0x27f0 fs/namei.c:3507
do_filp_open+0x1aa/0x400 fs/namei.c:3534
do_sys_openat2+0x16d/0x420 fs/open.c:1204
do_sys_open fs/open.c:1220 [inline]
__do_sys_open fs/open.c:1228 [inline]
__se_sys_open fs/open.c:1224 [inline]
__x64_sys_open+0x119/0x1c0 fs/open.c:1224
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
other info that might help us debug this:
Chain exists of:
loop_ctl_mutex --> mtd_table_mutex --> major_names_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(major_names_lock);
lock(mtd_table_mutex);
lock(major_names_lock);
lock(loop_ctl_mutex);
*** DEADLOCK ***
1 lock held by systemd-udevd/8674:
#0: ffffffff8c1f3e08 (major_names_lock){+.+.}-{3:3}, at: blk_request_module+0x25/0x1d0 block/genhd.c:657
stack backtrace:
CPU: 0 PID: 8674 Comm: systemd-udevd Not tainted 5.13.0-next-20210707-syzkaller #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011
Call Trace:
__dump_stack lib/dump_stack.c:88 [inline]
dump_stack_lvl+0xcd/0x134 lib/dump_stack.c:105
check_noncircular+0x25f/0x2e0 kernel/locking/lockdep.c:2131
check_prev_add kernel/locking/lockdep.c:3051 [inline]
check_prevs_add kernel/locking/lockdep.c:3174 [inline]
validate_chain kernel/locking/lockdep.c:3789 [inline]
__lock_acquire+0x2a07/0x54a0 kernel/locking/lockdep.c:5015
lock_acquire kernel/locking/lockdep.c:5625 [inline]
lock_acquire+0x1ab/0x510 kernel/locking/lockdep.c:5590
__mutex_lock_common kernel/locking/mutex.c:959 [inline]
__mutex_lock+0x12a/0x10a0 kernel/locking/mutex.c:1104
loop_add+0x9c/0x8c0 drivers/block/loop.c:2250
loop_probe+0x6a/0x80 drivers/block/loop.c:2360
blk_request_module+0x111/0x1d0 block/genhd.c:660
blkdev_get_no_open+0x1d5/0x250 fs/block_dev.c:1332
blkdev_get_by_dev.part.0+0x25/0xdd0 fs/block_dev.c:1395
blkdev_get_by_dev fs/block_dev.c:1504 [inline]
blkdev_open+0x295/0x300 fs/block_dev.c:1510
do_dentry_open+0x4c8/0x11d0 fs/open.c:826
do_open fs/namei.c:3374 [inline]
path_openat+0x1c23/0x27f0 fs/namei.c:3507
do_filp_open+0x1aa/0x400 fs/namei.c:3534
do_sys_openat2+0x16d/0x420 fs/open.c:1204
do_sys_open fs/open.c:1220 [inline]
__do_sys_open fs/open.c:1228 [inline]
__se_sys_open fs/open.c:1224 [inline]
__x64_sys_open+0x119/0x1c0 fs/open.c:1224
do_syscall_x64 arch/x86/entry/common.c:50 [inline]
do_syscall_64+0x35/0xb0 arch/x86/entry/common.c:80
entry_SYSCALL_64_after_hwframe+0x44/0xae
RIP: 0033:0x7fb26e980840
Code: 73 01 c3 48 8b 0d 68 77 20 00 f7 d8 64 89 01 48 83 c8 ff c3 66 0f 1f 44 00 00 83 3d 89 bb 20 00 00 75 10 b8 02 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 31 c3 48 83 ec 08 e8 1e f6 ff ff 48 89 04 24
RSP: 002b:00007fff25793e98 EFLAGS: 00000246 ORIG_RAX: 0000000000000002
RAX: ffffffffffffffda RBX: 0000563b22a5f910 RCX: 00007fb26e980840
RDX: 0000563b21ce5fe3 RSI: 00000000000a0800 RDI: 0000563b22a5f850
RBP: 00007fff25794010 R08: 0000563b21ce5670 R09: 0000000000000010
R10: 0000563b21ce5d0c R11: 0000000000000246 R12: 00007fff25793f60
R13: 0000563b22a61030 R14: 0000000000000003 R15: 000000000000000e
---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.
syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.
For information about bisection process see: https://goo.gl/tpsmEJ#bisection
syzbot can test patches for this issue, for details see:
https://goo.gl/tpsmEJ#testing-patches
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [syzbot] possible deadlock in loop_add
[not found] ` <20210710131638.605-1-hdanton@sina.com>
@ 2021-07-12 5:27 ` Christoph Hellwig
2021-07-12 8:29 ` Desmond Cheong Zhi Xi
0 siblings, 1 reply; 5+ messages in thread
From: Christoph Hellwig @ 2021-07-12 5:27 UTC (permalink / raw)
To: Hillf Danton
Cc: syzbot, axboe, hch, linux-block, linux-kernel, syzkaller-bugs,
Desmond Cheong Zhi Xi
On Sat, Jul 10, 2021 at 09:16:38PM +0800, Hillf Danton wrote:
> To break the lock chain, un/register blkdev without mtd_table_mutex held.
Yes, Desmond Cheong Zhi Xi sent pretty much the same patch on June 18th
(mtd: break circular locks in register_mtd_blktrans), but it did not get
picked up.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [syzbot] possible deadlock in loop_add
2021-07-12 5:27 ` Christoph Hellwig
@ 2021-07-12 8:29 ` Desmond Cheong Zhi Xi
2021-07-15 23:00 ` Miquel Raynal
0 siblings, 1 reply; 5+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-07-12 8:29 UTC (permalink / raw)
To: Christoph Hellwig, Hillf Danton
Cc: syzbot, axboe, linux-block, linux-kernel, syzkaller-bugs, Miquel Raynal
On 12/7/21 1:27 pm, Christoph Hellwig wrote:
> On Sat, Jul 10, 2021 at 09:16:38PM +0800, Hillf Danton wrote:
>> To break the lock chain, un/register blkdev without mtd_table_mutex held.
>
> Yes, Desmond Cheong Zhi Xi sent pretty much the same patch on June 18th
> (mtd: break circular locks in register_mtd_blktrans), but it did not get
> picked up.
>
I believe Miquèl was waiting for -rc1 to apply it.
But taking a closer look, although the fix for the register path is the
same, Hillf Danton's proposed patch additionally avoids inverting the
lock hierarchy on the unregister path. So I believe this new patch
should be more robust.
Best wishes,
Desmond
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [syzbot] possible deadlock in loop_add
2021-07-12 8:29 ` Desmond Cheong Zhi Xi
@ 2021-07-15 23:00 ` Miquel Raynal
2021-07-17 9:41 ` Desmond Cheong Zhi Xi
0 siblings, 1 reply; 5+ messages in thread
From: Miquel Raynal @ 2021-07-15 23:00 UTC (permalink / raw)
To: Desmond Cheong Zhi Xi
Cc: Christoph Hellwig, Hillf Danton, syzbot, axboe, linux-block,
linux-kernel, syzkaller-bugs
Hello,
Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com> wrote on Mon, 12 Jul
2021 16:29:16 +0800:
> On 12/7/21 1:27 pm, Christoph Hellwig wrote:
> > On Sat, Jul 10, 2021 at 09:16:38PM +0800, Hillf Danton wrote:
> >> To break the lock chain, un/register blkdev without mtd_table_mutex held.
> >
> > Yes, Desmond Cheong Zhi Xi sent pretty much the same patch on June 18th
> > (mtd: break circular locks in register_mtd_blktrans), but it did not get
> > picked up.
> >
>
> I believe Miquèl was waiting for -rc1 to apply it.
Indeed, I already applied it but did not advertise yet.
>
> But taking a closer look, although the fix for the register path is the same, Hillf Danton's proposed patch additionally avoids inverting the lock hierarchy on the unregister path. So I believe this new patch should be more robust.
We can definitely do this in two steps if you want.
Thanks,
Miquèl
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [syzbot] possible deadlock in loop_add
2021-07-15 23:00 ` Miquel Raynal
@ 2021-07-17 9:41 ` Desmond Cheong Zhi Xi
0 siblings, 0 replies; 5+ messages in thread
From: Desmond Cheong Zhi Xi @ 2021-07-17 9:41 UTC (permalink / raw)
To: Miquel Raynal, Hillf Danton
Cc: Christoph Hellwig, syzbot, axboe, linux-block, linux-kernel,
syzkaller-bugs
On 16/7/21 7:00 am, Miquel Raynal wrote:
> Hello,
>
> Desmond Cheong Zhi Xi <desmondcheongzx@gmail.com> wrote on Mon, 12 Jul
> 2021 16:29:16 +0800:
>
>> On 12/7/21 1:27 pm, Christoph Hellwig wrote:
>>> On Sat, Jul 10, 2021 at 09:16:38PM +0800, Hillf Danton wrote:
>>>> To break the lock chain, un/register blkdev without mtd_table_mutex held.
>>>
>>> Yes, Desmond Cheong Zhi Xi sent pretty much the same patch on June 18th
>>> (mtd: break circular locks in register_mtd_blktrans), but it did not get
>>> picked up.
>>>
>>
>> I believe Miquèl was waiting for -rc1 to apply it.
>
> Indeed, I already applied it but did not advertise yet.
>
Thanks Miquèl!
>>
>> But taking a closer look, although the fix for the register path is the same, Hillf Danton's proposed patch additionally avoids inverting the lock hierarchy on the unregister path. So I believe this new patch should be more robust.
>
> We can definitely do this in two steps if you want.
>
Sounds good, I'll prepare a patch with Hillf's suggestion for the
unregister path.
> Thanks,
> Miquèl
>
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-07-17 9:41 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-07-10 11:01 [syzbot] possible deadlock in loop_add syzbot
[not found] ` <20210710131638.605-1-hdanton@sina.com>
2021-07-12 5:27 ` Christoph Hellwig
2021-07-12 8:29 ` Desmond Cheong Zhi Xi
2021-07-15 23:00 ` Miquel Raynal
2021-07-17 9:41 ` Desmond Cheong Zhi Xi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.