[syzbot] possible deadlock in map_mft_record

[syzbot] possible deadlock in map_mft_record

[syzbot]

Hello,

syzbot found the following issue on:

HEAD commit:    bbed346d5a96 Merge branch 'for-next/core' into for-kernelci
git tree:       git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux.git for-kernelci
console output: https://syzkaller.appspot.com/x/log.txt?x=17c6e19a880000
kernel config:  https://syzkaller.appspot.com/x/.config?x=3a4a45d2d827c1e
dashboard link: https://syzkaller.appspot.com/bug?extid=cb1fdea540b46f0ce394
compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
userspace arch: arm64

Unfortunately, I don't have any reproducer for this issue yet.

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/e8e91bc79312/disk-bbed346d.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/c1cb3fb3b77e/vmlinux-bbed346d.xz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+cb1fdea540b46f0ce394@syzkaller.appspotmail.com

ntfs: (device loop3): is_boot_sector_ntfs(): Invalid boot sector checksum.
ntfs: volume version 3.1.
======================================================
WARNING: possible circular locking dependency detected
6.0.0-rc7-syzkaller-18095-gbbed346d5a96 #0 Not tainted
------------------------------------------------------
syz-executor.3/9828 is trying to acquire lock:
ffff00010ca1f5d0 (&ni->mrec_lock){+.+.}-{3:3}, at: map_mft_record+0x40/0xfc fs/ntfs/mft.c:154

but task is already holding lock:
ffff00010ca1f540 (&rl->lock){++++}-{3:3}, at: ntfs_truncate+0xa8/0xf6c fs/ntfs/inode.c:2371

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&rl->lock){++++}-{3:3}:
       down_read+0x5c/0x78 kernel/locking/rwsem.c:1499
       ntfs_read_block+0x2d0/0x1110 fs/ntfs/aops.c:248
       ntfs_read_folio+0x594/0x758 fs/ntfs/aops.c:436
       filemap_read_folio+0x68/0x33c mm/filemap.c:2394
       do_read_cache_folio+0x1c8/0x588 mm/filemap.c:3519
       do_read_cache_page mm/filemap.c:3561 [inline]
       read_cache_page+0x40/0x178 mm/filemap.c:3570
       read_mapping_page include/linux/pagemap.h:756 [inline]
       ntfs_map_page fs/ntfs/aops.h:75 [inline]
       ntfs_sync_mft_mirror+0xb4/0x109c fs/ntfs/mft.c:480
       write_mft_record_nolock+0x814/0xc70 fs/ntfs/mft.c:787
       write_mft_record+0xe4/0x230 fs/ntfs/mft.h:95
       __ntfs_write_inode+0x32c/0x554 fs/ntfs/inode.c:3043
       ntfs_write_inode+0x3c/0x4c fs/ntfs/super.c:2655
       write_inode fs/fs-writeback.c:1440 [inline]
       __writeback_single_inode+0x240/0x2e4 fs/fs-writeback.c:1652
       writeback_sb_inodes+0x3e4/0x85c fs/fs-writeback.c:1865
       wb_writeback+0x198/0x328 fs/fs-writeback.c:2039
       wb_do_writeback+0xc8/0x384 fs/fs-writeback.c:2182
       wb_workfn+0x70/0x15c fs/fs-writeback.c:2222
       process_one_work+0x2d8/0x504 kernel/workqueue.c:2289
       worker_thread+0x340/0x610 kernel/workqueue.c:2436
       kthread+0x12c/0x158 kernel/kthread.c:376
       ret_from_fork+0x10/0x20 arch/arm64/kernel/entry.S:860

-> #0 (&ni->mrec_lock){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3095 [inline]
       check_prevs_add kernel/locking/lockdep.c:3214 [inline]
       validate_chain kernel/locking/lockdep.c:3829 [inline]
       __lock_acquire+0x1530/0x30a4 kernel/locking/lockdep.c:5053
       lock_acquire+0x100/0x1f8 kernel/locking/lockdep.c:5666
       __mutex_lock_common+0xd4/0xca8 kernel/locking/mutex.c:603
       __mutex_lock kernel/locking/mutex.c:747 [inline]
       mutex_lock_nested+0x38/0x44 kernel/locking/mutex.c:799
       map_mft_record+0x40/0xfc fs/ntfs/mft.c:154
       ntfs_truncate+0xd8/0xf6c fs/ntfs/inode.c:2376
       ntfs_truncate_vfs fs/ntfs/inode.c:2855 [inline]
       ntfs_setattr+0x1ac/0x228 fs/ntfs/inode.c:2907
       notify_change+0x758/0x7f0 fs/attr.c:420
       do_truncate+0x108/0x150 fs/open.c:65
       vfs_truncate+0x1dc/0x24c fs/open.c:111
       do_sys_truncate+0x94/0x140 fs/open.c:134
       __do_sys_truncate fs/open.c:146 [inline]
       __se_sys_truncate fs/open.c:144 [inline]
       __arm64_sys_truncate+0x20/0x30 fs/open.c:144
       __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
       invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
       el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
       do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
       el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:636
       el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:654
       el0t_64_sync+0x18c/0x190 arch/arm64/kernel/entry.S:581

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&rl->lock);
                               lock(&ni->mrec_lock);
                               lock(&rl->lock);
  lock(&ni->mrec_lock);

 *** DEADLOCK ***

3 locks held by syz-executor.3/9828:
 #0: ffff0001067af460 (sb_writers#23){.+.+}-{0:0}, at: mnt_want_write+0x20/0x64 fs/namespace.c:393
 #1: ffff00010ca1f8b0 (&sb->s_type->i_mutex_key#32){+.+.}-{3:3}, at: inode_lock include/linux/fs.h:756 [inline]
 #1: ffff00010ca1f8b0 (&sb->s_type->i_mutex_key#32){+.+.}-{3:3}, at: do_truncate+0xf4/0x150 fs/open.c:63
 #2: ffff00010ca1f540 (&rl->lock){++++}-{3:3}, at: ntfs_truncate+0xa8/0xf6c fs/ntfs/inode.c:2371

stack backtrace:
CPU: 1 PID: 9828 Comm: syz-executor.3 Not tainted 6.0.0-rc7-syzkaller-18095-gbbed346d5a96 #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 09/30/2022
Call trace:
 dump_backtrace+0x1c4/0x1f0 arch/arm64/kernel/stacktrace.c:156
 show_stack+0x2c/0x54 arch/arm64/kernel/stacktrace.c:163
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x104/0x16c lib/dump_stack.c:106
 dump_stack+0x1c/0x58 lib/dump_stack.c:113
 print_circular_bug+0x2c4/0x2c8 kernel/locking/lockdep.c:2053
 check_noncircular+0x14c/0x154 kernel/locking/lockdep.c:2175
 check_prev_add kernel/locking/lockdep.c:3095 [inline]
 check_prevs_add kernel/locking/lockdep.c:3214 [inline]
 validate_chain kernel/locking/lockdep.c:3829 [inline]
 __lock_acquire+0x1530/0x30a4 kernel/locking/lockdep.c:5053
 lock_acquire+0x100/0x1f8 kernel/locking/lockdep.c:5666
 __mutex_lock_common+0xd4/0xca8 kernel/locking/mutex.c:603
 __mutex_lock kernel/locking/mutex.c:747 [inline]
 mutex_lock_nested+0x38/0x44 kernel/locking/mutex.c:799
 map_mft_record+0x40/0xfc fs/ntfs/mft.c:154
 ntfs_truncate+0xd8/0xf6c fs/ntfs/inode.c:2376
 ntfs_truncate_vfs fs/ntfs/inode.c:2855 [inline]
 ntfs_setattr+0x1ac/0x228 fs/ntfs/inode.c:2907
 notify_change+0x758/0x7f0 fs/attr.c:420
 do_truncate+0x108/0x150 fs/open.c:65
 vfs_truncate+0x1dc/0x24c fs/open.c:111
 do_sys_truncate+0x94/0x140 fs/open.c:134
 __do_sys_truncate fs/open.c:146 [inline]
 __se_sys_truncate fs/open.c:144 [inline]
 __arm64_sys_truncate+0x20/0x30 fs/open.c:144
 __invoke_syscall arch/arm64/kernel/syscall.c:38 [inline]
 invoke_syscall arch/arm64/kernel/syscall.c:52 [inline]
 el0_svc_common+0x138/0x220 arch/arm64/kernel/syscall.c:142
 do_el0_svc+0x48/0x164 arch/arm64/kernel/syscall.c:206
 el0_svc+0x58/0x150 arch/arm64/kernel/entry-common.c:636
 el0t_64_sync_handler+0x84/0xf0 arch/arm64/kernel/entry-common.c:654
 el0t_64_sync+0x18c/0x190 arch/arm64/kernel/entry.S:581


---
This report is generated by a bot. It may contain errors.
See https://goo.gl/tpsmEJ for more information about syzbot.
syzbot engineers can be reached at syzkaller@googlegroups.com.

syzbot will keep track of this issue. See:
https://goo.gl/tpsmEJ#status for how to communicate with syzbot.

Re: [syzbot] [ntfs?] possible deadlock in map_mft_record

[syzbot]

syzbot has found a reproducer for the following issue on:

HEAD commit:    2258c2dc850b Merge tag 'for-linus' of git://git.kernel.org..
git tree:       upstream
console+strace: https://syzkaller.appspot.com/x/log.txt?x=1285d048480000
kernel config:  https://syzkaller.appspot.com/x/.config?x=68e0be42c8ee4bb4
dashboard link: https://syzkaller.appspot.com/bug?extid=cb1fdea540b46f0ce394
compiler:       Debian clang version 13.0.1-++20220126092033+75e33f71c2da-1~exp1~20220126212112.63, GNU ld (GNU Binutils for Debian) 2.35.2
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=15d788b8480000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=15347b32480000

Downloadable assets:
disk image: https://storage.googleapis.com/syzbot-assets/aee4b2292a64/disk-2258c2dc.raw.xz
vmlinux: https://storage.googleapis.com/syzbot-assets/9ecfc816182c/vmlinux-2258c2dc.xz
kernel image: https://storage.googleapis.com/syzbot-assets/f24ffaf1255a/bzImage-2258c2dc.xz
mounted in repro: https://storage.googleapis.com/syzbot-assets/0f6a807498b6/mount_0.gz

IMPORTANT: if you fix the issue, please add the following tag to the commit:
Reported-by: syzbot+cb1fdea540b46f0ce394@syzkaller.appspotmail.com

loop0: detected capacity change from 0 to 4096
ntfs: volume version 3.1.
======================================================
WARNING: possible circular locking dependency detected
6.2.0-rc1-syzkaller-00043-g2258c2dc850b #0 Not tainted
------------------------------------------------------
syz-executor154/5075 is trying to acquire lock:
ffff888073152ad0 (&lcnbmp_mrec_lock_key){+.+.}-{3:3}, at: map_mft_record+0x46/0x610 fs/ntfs/mft.c:154

but task is already holding lock:
ffff88802ac261f8 (&vol->lcnbmp_lock){+.+.}-{3:3}, at: ntfs_put_super+0x36b/0xf80 fs/ntfs/super.c:2282

which lock already depends on the new lock.


the existing dependency chain (in reverse order) is:

-> #1 (&vol->lcnbmp_lock){+.+.}-{3:3}:
       lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
       down_write+0x9c/0x270 kernel/locking/rwsem.c:1562
       __ntfs_cluster_free+0xd4/0x890 fs/ntfs/lcnalloc.c:862
       ntfs_cluster_free fs/ntfs/lcnalloc.h:96 [inline]
       ntfs_truncate+0x119c/0x2720 fs/ntfs/inode.c:2695
       ntfs_truncate_vfs fs/ntfs/inode.c:2862 [inline]
       ntfs_setattr+0x2b9/0x3a0 fs/ntfs/inode.c:2914
       notify_change+0xe50/0x1100 fs/attr.c:482
       do_truncate+0x200/0x2f0 fs/open.c:65
       handle_truncate fs/namei.c:3216 [inline]
       do_open fs/namei.c:3561 [inline]
       path_openat+0x272b/0x2dd0 fs/namei.c:3714
       do_file_open_root+0x339/0x790 fs/namei.c:3766
       file_open_root+0x234/0x290 fs/open.c:1290
       do_handle_open+0x565/0x950 fs/fhandle.c:232
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

-> #0 (&lcnbmp_mrec_lock_key){+.+.}-{3:3}:
       check_prev_add kernel/locking/lockdep.c:3097 [inline]
       check_prevs_add kernel/locking/lockdep.c:3216 [inline]
       validate_chain+0x1898/0x6ae0 kernel/locking/lockdep.c:3831
       __lock_acquire+0x1292/0x1f60 kernel/locking/lockdep.c:5055
       lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
       __mutex_lock_common+0x1bd/0x26e0 kernel/locking/mutex.c:603
       __mutex_lock kernel/locking/mutex.c:747 [inline]
       mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799
       map_mft_record+0x46/0x610 fs/ntfs/mft.c:154
       __ntfs_write_inode+0x80/0xc90 fs/ntfs/inode.c:2978
       ntfs_commit_inode fs/ntfs/inode.h:300 [inline]
       ntfs_put_super+0x3ba/0xf80 fs/ntfs/super.c:2283
       generic_shutdown_super+0x130/0x310 fs/super.c:492
       kill_block_super+0x79/0xd0 fs/super.c:1386
       deactivate_locked_super+0xa7/0xf0 fs/super.c:332
       cleanup_mnt+0x494/0x520 fs/namespace.c:1291
       task_work_run+0x243/0x300 kernel/task_work.c:179
       exit_task_work include/linux/task_work.h:38 [inline]
       do_exit+0x644/0x2150 kernel/exit.c:867
       do_group_exit+0x1fd/0x2b0 kernel/exit.c:1012
       __do_sys_exit_group kernel/exit.c:1023 [inline]
       __se_sys_exit_group kernel/exit.c:1021 [inline]
       __x64_sys_exit_group+0x3b/0x40 kernel/exit.c:1021
       do_syscall_x64 arch/x86/entry/common.c:50 [inline]
       do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
       entry_SYSCALL_64_after_hwframe+0x63/0xcd

other info that might help us debug this:

 Possible unsafe locking scenario:

       CPU0                    CPU1
       ----                    ----
  lock(&vol->lcnbmp_lock);
                               lock(&lcnbmp_mrec_lock_key);
                               lock(&vol->lcnbmp_lock);
  lock(&lcnbmp_mrec_lock_key);

 *** DEADLOCK ***

2 locks held by syz-executor154/5075:
 #0: ffff88807a03e0e0 (&type->s_umount_key#46){+.+.}-{3:3}, at: deactivate_super+0x96/0xd0 fs/super.c:362
 #1: ffff88802ac261f8 (&vol->lcnbmp_lock){+.+.}-{3:3}, at: ntfs_put_super+0x36b/0xf80 fs/ntfs/super.c:2282

stack backtrace:
CPU: 1 PID: 5075 Comm: syz-executor154 Not tainted 6.2.0-rc1-syzkaller-00043-g2258c2dc850b #0
Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 10/26/2022
Call Trace:
 <TASK>
 __dump_stack lib/dump_stack.c:88 [inline]
 dump_stack_lvl+0x1b1/0x290 lib/dump_stack.c:106
 check_noncircular+0x2cc/0x390 kernel/locking/lockdep.c:2177
 check_prev_add kernel/locking/lockdep.c:3097 [inline]
 check_prevs_add kernel/locking/lockdep.c:3216 [inline]
 validate_chain+0x1898/0x6ae0 kernel/locking/lockdep.c:3831
 __lock_acquire+0x1292/0x1f60 kernel/locking/lockdep.c:5055
 lock_acquire+0x182/0x3c0 kernel/locking/lockdep.c:5668
 __mutex_lock_common+0x1bd/0x26e0 kernel/locking/mutex.c:603
 __mutex_lock kernel/locking/mutex.c:747 [inline]
 mutex_lock_nested+0x17/0x20 kernel/locking/mutex.c:799
 map_mft_record+0x46/0x610 fs/ntfs/mft.c:154
 __ntfs_write_inode+0x80/0xc90 fs/ntfs/inode.c:2978
 ntfs_commit_inode fs/ntfs/inode.h:300 [inline]
 ntfs_put_super+0x3ba/0xf80 fs/ntfs/super.c:2283
 generic_shutdown_super+0x130/0x310 fs/super.c:492
 kill_block_super+0x79/0xd0 fs/super.c:1386
 deactivate_locked_super+0xa7/0xf0 fs/super.c:332
 cleanup_mnt+0x494/0x520 fs/namespace.c:1291
 task_work_run+0x243/0x300 kernel/task_work.c:179
 exit_task_work include/linux/task_work.h:38 [inline]
 do_exit+0x644/0x2150 kernel/exit.c:867
 do_group_exit+0x1fd/0x2b0 kernel/exit.c:1012
 __do_sys_exit_group kernel/exit.c:1023 [inline]
 __se_sys_exit_group kernel/exit.c:1021 [inline]
 __x64_sys_exit_group+0x3b/0x40 kernel/exit.c:1021
 do_syscall_x64 arch/x86/entry/common.c:50 [inline]
 do_syscall_64+0x3d/0xb0 arch/x86/entry/common.c:80
 entry_SYSCALL_64_after_hwframe+0x63/0xcd
RIP: 0033:0x7f1316ebea09
Code: Unable to access opcode bytes at 0x7f1316ebe9df.
RSP: 002b:00007ffe53bdb118 EFLAGS: 0000

Re: [syzbot] [ntfs3?] possible deadlock in map_mft_record

[syzbot]

syzbot suspects this issue was fixed by commit:

commit 6f861765464f43a71462d52026fbddfc858239a5
Author: Jan Kara <jack@suse.cz>
Date:   Wed Nov 1 17:43:10 2023 +0000

    fs: Block writes to mounted block devices

bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=12b28b99180000
start commit:   e4cf7c25bae5 Merge tag 'kbuild-fixes-v6.2' of git://git.ke..
git tree:       upstream
kernel config:  https://syzkaller.appspot.com/x/.config?x=68e0be42c8ee4bb4
dashboard link: https://syzkaller.appspot.com/bug?extid=cb1fdea540b46f0ce394
syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=151db82a480000
C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=10f0e670480000

If the result looks correct, please mark the issue as fixed by replying with:

#syz fix: fs: Block writes to mounted block devices

For information about bisection process see: https://goo.gl/tpsmEJ#bisection

Re: [syzbot] [ntfs3?] possible deadlock in map_mft_record

[Jan Kara]

On Sat 09-03-24 18:05:04, syzbot wrote:
> syzbot suspects this issue was fixed by commit:
> 
> commit 6f861765464f43a71462d52026fbddfc858239a5
> Author: Jan Kara <jack@suse.cz>
> Date:   Wed Nov 1 17:43:10 2023 +0000
> 
>     fs: Block writes to mounted block devices
> 
> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=12b28b99180000
> start commit:   e4cf7c25bae5 Merge tag 'kbuild-fixes-v6.2' of git://git.ke..
> git tree:       upstream
> kernel config:  https://syzkaller.appspot.com/x/.config?x=68e0be42c8ee4bb4
> dashboard link: https://syzkaller.appspot.com/bug?extid=cb1fdea540b46f0ce394
> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=151db82a480000
> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=10f0e670480000
> 
> If the result looks correct, please mark the issue as fixed by replying with:
 
Looks good.

#syz fix: fs: Block writes to mounted block devices

								Honza
-- 
Jan Kara <jack@suse.com>
SUSE Labs, CR