* possible deadlock in vfs_fallocate @ 2018-04-29 17:00 syzbot 2018-05-09 7:57 ` Eric Biggers 0 siblings, 1 reply; 4+ messages in thread From: syzbot @ 2018-04-29 17:00 UTC (permalink / raw) To: linux-fsdevel, linux-kernel, syzkaller-bugs, viro Hello, syzbot hit the following crash on upstream commit cdface5209349930ae1b51338763c8e029971b97 (Sun Apr 29 03:07:21 2018 +0000) Merge tag 'for_linus_stable' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4 syzbot dashboard link: https://syzkaller.appspot.com/bug?extid=148c2885d71194f18d28 C reproducer: https://syzkaller.appspot.com/x/repro.c?id=5054004375584768 syzkaller reproducer: https://syzkaller.appspot.com/x/repro.syz?id=6438048191479808 Raw console output: https://syzkaller.appspot.com/x/log.txt?id=5404215203594240 Kernel config: https://syzkaller.appspot.com/x/.config?id=7043958930931867332 compiler: gcc (GCC) 8.0.1 20180413 (experimental) IMPORTANT: if you fix the bug, please add the following tag to the commit: Reported-by: syzbot+148c2885d71194f18d28@syzkaller.appspotmail.com It will help syzbot understand when the bug is fixed. See footer for details. If you forward the report, please keep this part and the footer. random: sshd: uninitialized urandom read (32 bytes read) random: sshd: uninitialized urandom read (32 bytes read) random: sshd: uninitialized urandom read (32 bytes read) ====================================================== WARNING: possible circular locking dependency detected 4.17.0-rc2+ #23 Not tainted ------------------------------------------------------ syz-executor715/4492 is trying to acquire lock: (ptrval) (sb_writers#6){.+.+}, at: file_start_write include/linux/fs.h:2718 [inline] (ptrval) (sb_writers#6){.+.+}, at: vfs_fallocate+0x5be/0x8d0 fs/open.c:318 but task is already holding lock: (ptrval) (ashmem_mutex){+.+.}, at: ashmem_shrink_scan+0xac/0x560 drivers/staging/android/ashmem.c:440 which lock already depends on the new lock. the existing dependency chain (in reverse order) is: -> #3 (ashmem_mutex){+.+.}: __mutex_lock_common kernel/locking/mutex.c:756 [inline] __mutex_lock+0x16d/0x17f0 kernel/locking/mutex.c:893 mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908 ashmem_mmap+0x53/0x460 drivers/staging/android/ashmem.c:361 call_mmap include/linux/fs.h:1789 [inline] mmap_region+0xd13/0x1820 mm/mmap.c:1723 do_mmap+0xc79/0x11d0 mm/mmap.c:1494 do_mmap_pgoff include/linux/mm.h:2237 [inline] vm_mmap_pgoff+0x1fb/0x2a0 mm/util.c:357 ksys_mmap_pgoff+0x4c9/0x640 mm/mmap.c:1544 __do_sys_mmap arch/x86/kernel/sys_x86_64.c:100 [inline] __se_sys_mmap arch/x86/kernel/sys_x86_64.c:91 [inline] __x64_sys_mmap+0xe9/0x1b0 arch/x86/kernel/sys_x86_64.c:91 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe -> #2 (&mm->mmap_sem){++++}: __might_fault+0x155/0x1e0 mm/memory.c:4555 _copy_to_user+0x30/0x110 lib/usercopy.c:25 copy_to_user include/linux/uaccess.h:155 [inline] filldir+0x1ea/0x3a0 fs/readdir.c:196 dir_emit_dot include/linux/fs.h:3378 [inline] dir_emit_dots include/linux/fs.h:3389 [inline] dcache_readdir+0x13a/0x620 fs/libfs.c:192 iterate_dir+0x4b0/0x5d0 fs/readdir.c:51 __do_sys_getdents fs/readdir.c:231 [inline] __se_sys_getdents fs/readdir.c:212 [inline] __x64_sys_getdents+0x293/0x4e0 fs/readdir.c:212 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe -> #1 (&sb->s_type->i_mutex_key#11){++++}: down_write+0x87/0x120 kernel/locking/rwsem.c:70 inode_lock include/linux/fs.h:713 [inline] do_last fs/namei.c:3274 [inline] path_openat+0x123b/0x4e20 fs/namei.c:3501 do_filp_open+0x249/0x350 fs/namei.c:3535 do_sys_open+0x56f/0x740 fs/open.c:1093 __do_sys_open fs/open.c:1111 [inline] __se_sys_open fs/open.c:1106 [inline] __x64_sys_open+0x7e/0xc0 fs/open.c:1106 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe -> #0 (sb_writers#6){.+.+}: lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:3920 percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline] percpu_down_read include/linux/percpu-rwsem.h:59 [inline] __sb_start_write+0x1e9/0x300 fs/super.c:1385 file_start_write include/linux/fs.h:2718 [inline] vfs_fallocate+0x5be/0x8d0 fs/open.c:318 ashmem_shrink_scan+0x1f1/0x560 drivers/staging/android/ashmem.c:447 ashmem_ioctl+0x3bf/0x13a0 drivers/staging/android/ashmem.c:789 vfs_ioctl fs/ioctl.c:46 [inline] file_ioctl fs/ioctl.c:500 [inline] do_vfs_ioctl+0x1cf/0x16a0 fs/ioctl.c:684 ksys_ioctl+0xa9/0xd0 fs/ioctl.c:701 __do_sys_ioctl fs/ioctl.c:708 [inline] __se_sys_ioctl fs/ioctl.c:706 [inline] __x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:706 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe other info that might help us debug this: Chain exists of: sb_writers#6 --> &mm->mmap_sem --> ashmem_mutex Possible unsafe locking scenario: CPU0 CPU1 ---- ---- lock(ashmem_mutex); lock(&mm->mmap_sem); lock(ashmem_mutex); lock(sb_writers#6); *** DEADLOCK *** 1 lock held by syz-executor715/4492: #0: (ptrval) (ashmem_mutex){+.+.}, at: ashmem_shrink_scan+0xac/0x560 drivers/staging/android/ashmem.c:440 stack backtrace: CPU: 1 PID: 4492 Comm: syz-executor715 Not tainted 4.17.0-rc2+ #23 Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS Google 01/01/2011 Call Trace: __dump_stack lib/dump_stack.c:77 [inline] dump_stack+0x1b9/0x294 lib/dump_stack.c:113 print_circular_bug.isra.36.cold.54+0x1bd/0x27d kernel/locking/lockdep.c:1223 check_prev_add kernel/locking/lockdep.c:1863 [inline] check_prevs_add kernel/locking/lockdep.c:1976 [inline] validate_chain kernel/locking/lockdep.c:2417 [inline] __lock_acquire+0x343e/0x5140 kernel/locking/lockdep.c:3431 lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:3920 percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline] percpu_down_read include/linux/percpu-rwsem.h:59 [inline] __sb_start_write+0x1e9/0x300 fs/super.c:1385 file_start_write include/linux/fs.h:2718 [inline] vfs_fallocate+0x5be/0x8d0 fs/open.c:318 ashmem_shrink_scan+0x1f1/0x560 drivers/staging/android/ashmem.c:447 ashmem_ioctl+0x3bf/0x13a0 drivers/staging/android/ashmem.c:789 vfs_ioctl fs/ioctl.c:46 [inline] file_ioctl fs/ioctl.c:500 [inline] do_vfs_ioctl+0x1cf/0x16a0 fs/ioctl.c:684 ksys_ioctl+0xa9/0xd0 fs/ioctl.c:701 __do_sys_ioctl fs/ioctl.c:708 [inline] __se_sys_ioctl fs/ioctl.c:706 [inline] __x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:706 do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 entry_SYSCALL_64_after_hwframe+0x49/0xbe RIP: 0033:0x440179 RSP: 002b:00007ffc165d4a28 EFLAGS: 00000217 ORIG_RAX: 0000000000000010 RAX: ffffffffffffffda RBX: 6873612f7665642f RCX: 0000000000440179 RDX: 0000000000000000 RSI: 000000000000770a RDI: 0000000000000004 RBP: 00000000006ca018 R08: 0000000000000000 R09: 0000000000000000 R10: 0000000000000004 R11: 0000000000000217 R12: 00000000004016a0 R13: 0000000000401730 R14: 0000000000000000 R15: 0000000000000000 random: crng init done --- This bug is generated by a dumb bot. It may contain errors. See https://goo.gl/tpsmEJ for details. Direct all questions to syzkaller@googlegroups.com. syzbot will keep track of this bug report. If you forgot to add the Reported-by tag, once the fix for this bug is merged into any tree, please reply to this email with: #syz fix: exact-commit-title If you want to test a patch for this bug, please reply with: #syz test: git://repo/address.git branch and provide the patch inline or as an attachment. To mark this as a duplicate of another syzbot report, please reply with: #syz dup: exact-subject-of-another-report If it's a one-off invalid bug report, please reply with: #syz invalid Note: if the crash happens again, it will cause creation of a new bug report. Note: all commands must start from beginning of the line in the email body. ^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: possible deadlock in vfs_fallocate 2018-04-29 17:00 possible deadlock in vfs_fallocate syzbot @ 2018-05-09 7:57 ` Eric Biggers [not found] ` <CAHRSSEwAjrxZLRa3tPhjvz2MFiqVbCysTTgp6oKc=3Pcr1b2JQ@mail.gmail.com> 0 siblings, 1 reply; 4+ messages in thread From: Eric Biggers @ 2018-05-09 7:57 UTC (permalink / raw) To: Greg Kroah-Hartman, Arve Hjønnevåg, Todd Kjos, Martijn Coenen, devel Cc: linux-fsdevel, linux-kernel, syzkaller-bugs, syzbot [+ashmem maintainers] On Sun, Apr 29, 2018 at 10:00:03AM -0700, syzbot wrote: > Hello, > > syzbot hit the following crash on upstream commit > cdface5209349930ae1b51338763c8e029971b97 (Sun Apr 29 03:07:21 2018 +0000) > Merge tag 'for_linus_stable' of > git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4 > syzbot dashboard link: > https://syzkaller.appspot.com/bug?extid=148c2885d71194f18d28 > > C reproducer: https://syzkaller.appspot.com/x/repro.c?id=5054004375584768 > syzkaller reproducer: > https://syzkaller.appspot.com/x/repro.syz?id=6438048191479808 > Raw console output: > https://syzkaller.appspot.com/x/log.txt?id=5404215203594240 > Kernel config: > https://syzkaller.appspot.com/x/.config?id=7043958930931867332 > compiler: gcc (GCC) 8.0.1 20180413 (experimental) > > IMPORTANT: if you fix the bug, please add the following tag to the commit: > Reported-by: syzbot+148c2885d71194f18d28@syzkaller.appspotmail.com > It will help syzbot understand when the bug is fixed. See footer for > details. > If you forward the report, please keep this part and the footer. > > random: sshd: uninitialized urandom read (32 bytes read) > random: sshd: uninitialized urandom read (32 bytes read) > random: sshd: uninitialized urandom read (32 bytes read) > > ====================================================== > WARNING: possible circular locking dependency detected > 4.17.0-rc2+ #23 Not tainted > ------------------------------------------------------ > syz-executor715/4492 is trying to acquire lock: > (ptrval) (sb_writers#6){.+.+}, at: file_start_write > include/linux/fs.h:2718 [inline] > (ptrval) (sb_writers#6){.+.+}, at: vfs_fallocate+0x5be/0x8d0 > fs/open.c:318 > > but task is already holding lock: > (ptrval) (ashmem_mutex){+.+.}, at: ashmem_shrink_scan+0xac/0x560 > drivers/staging/android/ashmem.c:440 > > which lock already depends on the new lock. > > > the existing dependency chain (in reverse order) is: > > -> #3 (ashmem_mutex){+.+.}: > __mutex_lock_common kernel/locking/mutex.c:756 [inline] > __mutex_lock+0x16d/0x17f0 kernel/locking/mutex.c:893 > mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908 > ashmem_mmap+0x53/0x460 drivers/staging/android/ashmem.c:361 > call_mmap include/linux/fs.h:1789 [inline] > mmap_region+0xd13/0x1820 mm/mmap.c:1723 > do_mmap+0xc79/0x11d0 mm/mmap.c:1494 > do_mmap_pgoff include/linux/mm.h:2237 [inline] > vm_mmap_pgoff+0x1fb/0x2a0 mm/util.c:357 > ksys_mmap_pgoff+0x4c9/0x640 mm/mmap.c:1544 > __do_sys_mmap arch/x86/kernel/sys_x86_64.c:100 [inline] > __se_sys_mmap arch/x86/kernel/sys_x86_64.c:91 [inline] > __x64_sys_mmap+0xe9/0x1b0 arch/x86/kernel/sys_x86_64.c:91 > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > -> #2 (&mm->mmap_sem){++++}: > __might_fault+0x155/0x1e0 mm/memory.c:4555 > _copy_to_user+0x30/0x110 lib/usercopy.c:25 > copy_to_user include/linux/uaccess.h:155 [inline] > filldir+0x1ea/0x3a0 fs/readdir.c:196 > dir_emit_dot include/linux/fs.h:3378 [inline] > dir_emit_dots include/linux/fs.h:3389 [inline] > dcache_readdir+0x13a/0x620 fs/libfs.c:192 > iterate_dir+0x4b0/0x5d0 fs/readdir.c:51 > __do_sys_getdents fs/readdir.c:231 [inline] > __se_sys_getdents fs/readdir.c:212 [inline] > __x64_sys_getdents+0x293/0x4e0 fs/readdir.c:212 > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > -> #1 (&sb->s_type->i_mutex_key#11){++++}: > down_write+0x87/0x120 kernel/locking/rwsem.c:70 > inode_lock include/linux/fs.h:713 [inline] > do_last fs/namei.c:3274 [inline] > path_openat+0x123b/0x4e20 fs/namei.c:3501 > do_filp_open+0x249/0x350 fs/namei.c:3535 > do_sys_open+0x56f/0x740 fs/open.c:1093 > __do_sys_open fs/open.c:1111 [inline] > __se_sys_open fs/open.c:1106 [inline] > __x64_sys_open+0x7e/0xc0 fs/open.c:1106 > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > -> #0 (sb_writers#6){.+.+}: > lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:3920 > percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 > [inline] > percpu_down_read include/linux/percpu-rwsem.h:59 [inline] > __sb_start_write+0x1e9/0x300 fs/super.c:1385 > file_start_write include/linux/fs.h:2718 [inline] > vfs_fallocate+0x5be/0x8d0 fs/open.c:318 > ashmem_shrink_scan+0x1f1/0x560 drivers/staging/android/ashmem.c:447 > ashmem_ioctl+0x3bf/0x13a0 drivers/staging/android/ashmem.c:789 > vfs_ioctl fs/ioctl.c:46 [inline] > file_ioctl fs/ioctl.c:500 [inline] > do_vfs_ioctl+0x1cf/0x16a0 fs/ioctl.c:684 > ksys_ioctl+0xa9/0xd0 fs/ioctl.c:701 > __do_sys_ioctl fs/ioctl.c:708 [inline] > __se_sys_ioctl fs/ioctl.c:706 [inline] > __x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:706 > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > > other info that might help us debug this: > > Chain exists of: > sb_writers#6 --> &mm->mmap_sem --> ashmem_mutex > > Possible unsafe locking scenario: > > CPU0 CPU1 > ---- ---- > lock(ashmem_mutex); > lock(&mm->mmap_sem); > lock(ashmem_mutex); > lock(sb_writers#6); > > *** DEADLOCK *** > > 1 lock held by syz-executor715/4492: > #0: (ptrval) (ashmem_mutex){+.+.}, at: > ashmem_shrink_scan+0xac/0x560 drivers/staging/android/ashmem.c:440 > > stack backtrace: > CPU: 1 PID: 4492 Comm: syz-executor715 Not tainted 4.17.0-rc2+ #23 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS > Google 01/01/2011 > Call Trace: > __dump_stack lib/dump_stack.c:77 [inline] > dump_stack+0x1b9/0x294 lib/dump_stack.c:113 > print_circular_bug.isra.36.cold.54+0x1bd/0x27d > kernel/locking/lockdep.c:1223 > check_prev_add kernel/locking/lockdep.c:1863 [inline] > check_prevs_add kernel/locking/lockdep.c:1976 [inline] > validate_chain kernel/locking/lockdep.c:2417 [inline] > __lock_acquire+0x343e/0x5140 kernel/locking/lockdep.c:3431 > lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:3920 > percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline] > percpu_down_read include/linux/percpu-rwsem.h:59 [inline] > __sb_start_write+0x1e9/0x300 fs/super.c:1385 > file_start_write include/linux/fs.h:2718 [inline] > vfs_fallocate+0x5be/0x8d0 fs/open.c:318 > ashmem_shrink_scan+0x1f1/0x560 drivers/staging/android/ashmem.c:447 > ashmem_ioctl+0x3bf/0x13a0 drivers/staging/android/ashmem.c:789 > vfs_ioctl fs/ioctl.c:46 [inline] > file_ioctl fs/ioctl.c:500 [inline] > do_vfs_ioctl+0x1cf/0x16a0 fs/ioctl.c:684 > ksys_ioctl+0xa9/0xd0 fs/ioctl.c:701 > __do_sys_ioctl fs/ioctl.c:708 [inline] > __se_sys_ioctl fs/ioctl.c:706 [inline] > __x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:706 > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 > entry_SYSCALL_64_after_hwframe+0x49/0xbe > RIP: 0033:0x440179 > RSP: 002b:00007ffc165d4a28 EFLAGS: 00000217 ORIG_RAX: 0000000000000010 > RAX: ffffffffffffffda RBX: 6873612f7665642f RCX: 0000000000440179 > RDX: 0000000000000000 RSI: 000000000000770a RDI: 0000000000000004 > RBP: 00000000006ca018 R08: 0000000000000000 R09: 0000000000000000 > R10: 0000000000000004 R11: 0000000000000217 R12: 00000000004016a0 > R13: 0000000000401730 R14: 0000000000000000 R15: 0000000000000000 > random: crng init done > > > --- > This bug is generated by a dumb bot. It may contain errors. > See https://goo.gl/tpsmEJ for details. > Direct all questions to syzkaller@googlegroups.com. > > syzbot will keep track of this bug report. > If you forgot to add the Reported-by tag, once the fix for this bug is > merged > into any tree, please reply to this email with: > #syz fix: exact-commit-title > If you want to test a patch for this bug, please reply with: > #syz test: git://repo/address.git branch > and provide the patch inline or as an attachment. > To mark this as a duplicate of another syzbot report, please reply with: > #syz dup: exact-subject-of-another-report > If it's a one-off invalid bug report, please reply with: > #syz invalid > Note: if the crash happens again, it will cause creation of a new bug > report. > Note: all commands must start from beginning of the line in the email body. Looks like yet another locking bug in ashmem. ashmem_mutex normally ranks below mmap_sem, but ashmem_shrink_scan() inverts that order, via the file_start_write() in vfs_fallocate(). - Eric ^ permalink raw reply [flat|nested] 4+ messages in thread
[parent not found: <CAHRSSEwAjrxZLRa3tPhjvz2MFiqVbCysTTgp6oKc=3Pcr1b2JQ@mail.gmail.com>]
* Re: possible deadlock in vfs_fallocate [not found] ` <CAHRSSEwAjrxZLRa3tPhjvz2MFiqVbCysTTgp6oKc=3Pcr1b2JQ@mail.gmail.com> @ 2019-01-22 13:58 ` Dmitry Vyukov 2019-01-22 14:26 ` syzbot 0 siblings, 1 reply; 4+ messages in thread From: Dmitry Vyukov @ 2019-01-22 13:58 UTC (permalink / raw) To: Todd Kjos Cc: Eric Biggers, Joel Fernandes, Greg Kroah-Hartman, Arve Hjønnevåg, Todd Kjos, Martijn Coenen, open list:ANDROID DRIVERS, linux-fsdevel, LKML, syzkaller-bugs, syzbot+148c2885d71194f18d28, Tetsuo Handa [-- Attachment #1: Type: text/plain, Size: 9684 bytes --] On Wed, May 9, 2018 at 6:57 PM 'Todd Kjos' via syzkaller-bugs <syzkaller-bugs@googlegroups.com> wrote: > > +Joel Fernandes > > On Wed, May 9, 2018 at 12:55 AM Eric Biggers <ebiggers3@gmail.com> wrote: >> >> [+ashmem maintainers] >> >> On Sun, Apr 29, 2018 at 10:00:03AM -0700, syzbot wrote: >> > Hello, >> > >> > syzbot hit the following crash on upstream commit >> > cdface5209349930ae1b51338763c8e029971b97 (Sun Apr 29 03:07:21 2018 +0000) >> > Merge tag 'for_linus_stable' of >> > git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4 >> > syzbot dashboard link: >> > https://syzkaller.appspot.com/bug?extid=148c2885d71194f18d28 >> > >> > C reproducer: https://syzkaller.appspot.com/x/repro.c?id=5054004375584768 >> > syzkaller reproducer: >> > https://syzkaller.appspot.com/x/repro.syz?id=6438048191479808 >> > Raw console output: >> > https://syzkaller.appspot.com/x/log.txt?id=5404215203594240 >> > Kernel config: >> > https://syzkaller.appspot.com/x/.config?id=7043958930931867332 >> > compiler: gcc (GCC) 8.0.1 20180413 (experimental) Let's test Tetsuo's patch #syz test: git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git master >> > IMPORTANT: if you fix the bug, please add the following tag to the commit: >> > Reported-by: syzbot+148c2885d71194f18d28@syzkaller.appspotmail.com >> > It will help syzbot understand when the bug is fixed. See footer for >> > details. >> > If you forward the report, please keep this part and the footer. >> > >> > random: sshd: uninitialized urandom read (32 bytes read) >> > random: sshd: uninitialized urandom read (32 bytes read) >> > random: sshd: uninitialized urandom read (32 bytes read) >> > >> > ====================================================== >> > WARNING: possible circular locking dependency detected >> > 4.17.0-rc2+ #23 Not tainted >> > ------------------------------------------------------ >> > syz-executor715/4492 is trying to acquire lock: >> > (ptrval) (sb_writers#6){.+.+}, at: file_start_write >> > include/linux/fs.h:2718 [inline] >> > (ptrval) (sb_writers#6){.+.+}, at: vfs_fallocate+0x5be/0x8d0 >> > fs/open.c:318 >> > >> > but task is already holding lock: >> > (ptrval) (ashmem_mutex){+.+.}, at: ashmem_shrink_scan+0xac/0x560 >> > drivers/staging/android/ashmem.c:440 >> > >> > which lock already depends on the new lock. >> > >> > >> > the existing dependency chain (in reverse order) is: >> > >> > -> #3 (ashmem_mutex){+.+.}: >> > __mutex_lock_common kernel/locking/mutex.c:756 [inline] >> > __mutex_lock+0x16d/0x17f0 kernel/locking/mutex.c:893 >> > mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908 >> > ashmem_mmap+0x53/0x460 drivers/staging/android/ashmem.c:361 >> > call_mmap include/linux/fs.h:1789 [inline] >> > mmap_region+0xd13/0x1820 mm/mmap.c:1723 >> > do_mmap+0xc79/0x11d0 mm/mmap.c:1494 >> > do_mmap_pgoff include/linux/mm.h:2237 [inline] >> > vm_mmap_pgoff+0x1fb/0x2a0 mm/util.c:357 >> > ksys_mmap_pgoff+0x4c9/0x640 mm/mmap.c:1544 >> > __do_sys_mmap arch/x86/kernel/sys_x86_64.c:100 [inline] >> > __se_sys_mmap arch/x86/kernel/sys_x86_64.c:91 [inline] >> > __x64_sys_mmap+0xe9/0x1b0 arch/x86/kernel/sys_x86_64.c:91 >> > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 >> > entry_SYSCALL_64_after_hwframe+0x49/0xbe >> > >> > -> #2 (&mm->mmap_sem){++++}: >> > __might_fault+0x155/0x1e0 mm/memory.c:4555 >> > _copy_to_user+0x30/0x110 lib/usercopy.c:25 >> > copy_to_user include/linux/uaccess.h:155 [inline] >> > filldir+0x1ea/0x3a0 fs/readdir.c:196 >> > dir_emit_dot include/linux/fs.h:3378 [inline] >> > dir_emit_dots include/linux/fs.h:3389 [inline] >> > dcache_readdir+0x13a/0x620 fs/libfs.c:192 >> > iterate_dir+0x4b0/0x5d0 fs/readdir.c:51 >> > __do_sys_getdents fs/readdir.c:231 [inline] >> > __se_sys_getdents fs/readdir.c:212 [inline] >> > __x64_sys_getdents+0x293/0x4e0 fs/readdir.c:212 >> > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 >> > entry_SYSCALL_64_after_hwframe+0x49/0xbe >> > >> > -> #1 (&sb->s_type->i_mutex_key#11){++++}: >> > down_write+0x87/0x120 kernel/locking/rwsem.c:70 >> > inode_lock include/linux/fs.h:713 [inline] >> > do_last fs/namei.c:3274 [inline] >> > path_openat+0x123b/0x4e20 fs/namei.c:3501 >> > do_filp_open+0x249/0x350 fs/namei.c:3535 >> > do_sys_open+0x56f/0x740 fs/open.c:1093 >> > __do_sys_open fs/open.c:1111 [inline] >> > __se_sys_open fs/open.c:1106 [inline] >> > __x64_sys_open+0x7e/0xc0 fs/open.c:1106 >> > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 >> > entry_SYSCALL_64_after_hwframe+0x49/0xbe >> > >> > -> #0 (sb_writers#6){.+.+}: >> > lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:3920 >> > percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 >> > [inline] >> > percpu_down_read include/linux/percpu-rwsem.h:59 [inline] >> > __sb_start_write+0x1e9/0x300 fs/super.c:1385 >> > file_start_write include/linux/fs.h:2718 [inline] >> > vfs_fallocate+0x5be/0x8d0 fs/open.c:318 >> > ashmem_shrink_scan+0x1f1/0x560 drivers/staging/android/ashmem.c:447 >> > ashmem_ioctl+0x3bf/0x13a0 drivers/staging/android/ashmem.c:789 >> > vfs_ioctl fs/ioctl.c:46 [inline] >> > file_ioctl fs/ioctl.c:500 [inline] >> > do_vfs_ioctl+0x1cf/0x16a0 fs/ioctl.c:684 >> > ksys_ioctl+0xa9/0xd0 fs/ioctl.c:701 >> > __do_sys_ioctl fs/ioctl.c:708 [inline] >> > __se_sys_ioctl fs/ioctl.c:706 [inline] >> > __x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:706 >> > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 >> > entry_SYSCALL_64_after_hwframe+0x49/0xbe >> > >> > other info that might help us debug this: >> > >> > Chain exists of: >> > sb_writers#6 --> &mm->mmap_sem --> ashmem_mutex >> > >> > Possible unsafe locking scenario: >> > >> > CPU0 CPU1 >> > ---- ---- >> > lock(ashmem_mutex); >> > lock(&mm->mmap_sem); >> > lock(ashmem_mutex); >> > lock(sb_writers#6); >> > >> > *** DEADLOCK *** >> > >> > 1 lock held by syz-executor715/4492: >> > #0: (ptrval) (ashmem_mutex){+.+.}, at: >> > ashmem_shrink_scan+0xac/0x560 drivers/staging/android/ashmem.c:440 >> > >> > stack backtrace: >> > CPU: 1 PID: 4492 Comm: syz-executor715 Not tainted 4.17.0-rc2+ #23 >> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS >> > Google 01/01/2011 >> > Call Trace: >> > __dump_stack lib/dump_stack.c:77 [inline] >> > dump_stack+0x1b9/0x294 lib/dump_stack.c:113 >> > print_circular_bug.isra.36.cold.54+0x1bd/0x27d >> > kernel/locking/lockdep.c:1223 >> > check_prev_add kernel/locking/lockdep.c:1863 [inline] >> > check_prevs_add kernel/locking/lockdep.c:1976 [inline] >> > validate_chain kernel/locking/lockdep.c:2417 [inline] >> > __lock_acquire+0x343e/0x5140 kernel/locking/lockdep.c:3431 >> > lock_acquire+0x1dc/0x520 kernel/locking/lockdep.c:3920 >> > percpu_down_read_preempt_disable include/linux/percpu-rwsem.h:36 [inline] >> > percpu_down_read include/linux/percpu-rwsem.h:59 [inline] >> > __sb_start_write+0x1e9/0x300 fs/super.c:1385 >> > file_start_write include/linux/fs.h:2718 [inline] >> > vfs_fallocate+0x5be/0x8d0 fs/open.c:318 >> > ashmem_shrink_scan+0x1f1/0x560 drivers/staging/android/ashmem.c:447 >> > ashmem_ioctl+0x3bf/0x13a0 drivers/staging/android/ashmem.c:789 >> > vfs_ioctl fs/ioctl.c:46 [inline] >> > file_ioctl fs/ioctl.c:500 [inline] >> > do_vfs_ioctl+0x1cf/0x16a0 fs/ioctl.c:684 >> > ksys_ioctl+0xa9/0xd0 fs/ioctl.c:701 >> > __do_sys_ioctl fs/ioctl.c:708 [inline] >> > __se_sys_ioctl fs/ioctl.c:706 [inline] >> > __x64_sys_ioctl+0x73/0xb0 fs/ioctl.c:706 >> > do_syscall_64+0x1b1/0x800 arch/x86/entry/common.c:287 >> > entry_SYSCALL_64_after_hwframe+0x49/0xbe >> > RIP: 0033:0x440179 >> > RSP: 002b:00007ffc165d4a28 EFLAGS: 00000217 ORIG_RAX: 0000000000000010 >> > RAX: ffffffffffffffda RBX: 6873612f7665642f RCX: 0000000000440179 >> > RDX: 0000000000000000 RSI: 000000000000770a RDI: 0000000000000004 >> > RBP: 00000000006ca018 R08: 0000000000000000 R09: 0000000000000000 >> > R10: 0000000000000004 R11: 0000000000000217 R12: 00000000004016a0 >> > R13: 0000000000401730 R14: 0000000000000000 R15: 0000000000000000 >> > random: crng init done >> > >> > >> > --- >> > This bug is generated by a dumb bot. It may contain errors. >> > See https://goo.gl/tpsmEJ for details. >> > Direct all questions to syzkaller@googlegroups.com. >> > >> > syzbot will keep track of this bug report. >> > If you forgot to add the Reported-by tag, once the fix for this bug is >> > merged >> > into any tree, please reply to this email with: >> > #syz fix: exact-commit-title >> > If you want to test a patch for this bug, please reply with: >> > #syz test: git://repo/address.git branch >> > and provide the patch inline or as an attachment. >> > To mark this as a duplicate of another syzbot report, please reply with: >> > #syz dup: exact-subject-of-another-report >> > If it's a one-off invalid bug report, please reply with: >> > #syz invalid >> > Note: if the crash happens again, it will cause creation of a new bug >> > report. >> > Note: all commands must start from beginning of the line in the email body. >> >> Looks like yet another locking bug in ashmem. ashmem_mutex normally ranks below >> mmap_sem, but ashmem_shrink_scan() inverts that order, via the >> file_start_write() in vfs_fallocate(). >> >> - Eric [-- Attachment #2: ashmem.patch --] [-- Type: text/x-patch, Size: 3279 bytes --] diff --git a/drivers/staging/android/ashmem.c b/drivers/staging/android/ashmem.c index 90a8a9f1ac7d..1a890c43a10a 100644 --- a/drivers/staging/android/ashmem.c +++ b/drivers/staging/android/ashmem.c @@ -75,6 +75,17 @@ struct ashmem_range { /* LRU list of unpinned pages, protected by ashmem_mutex */ static LIST_HEAD(ashmem_lru_list); +static struct workqueue_struct *ashmem_wq; +static atomic_t ashmem_shrink_inflight = ATOMIC_INIT(0); +static DECLARE_WAIT_QUEUE_HEAD(ashmem_shrink_wait); + +struct ashmem_shrink_work { + struct work_struct work; + struct file *file; + loff_t start; + loff_t end; +}; + /* * long lru_count - The count of pages on our LRU list. * @@ -292,6 +303,7 @@ static ssize_t ashmem_read_iter(struct kiocb *iocb, struct iov_iter *iter) int ret = 0; mutex_lock(&ashmem_mutex); + wait_event(ashmem_shrink_wait, !atomic_read(&ashmem_shrink_inflight)); /* If size is not set, or set to 0, always return EOF. */ if (asma->size == 0) @@ -359,6 +371,7 @@ static int ashmem_mmap(struct file *file, struct vm_area_struct *vma) int ret = 0; mutex_lock(&ashmem_mutex); + wait_event(ashmem_shrink_wait, !atomic_read(&ashmem_shrink_inflight)); /* user needs to SET_SIZE before mapping */ if (!asma->size) { @@ -421,6 +434,19 @@ static int ashmem_mmap(struct file *file, struct vm_area_struct *vma) return ret; } +static void ashmem_shrink_worker(struct work_struct *work) +{ + struct ashmem_shrink_work *w = container_of(work, typeof(*w), work); + + w->file->f_op->fallocate(w->file, + FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, + w->start, w->end - w->start); + fput(w->file); + kfree(w); + if (atomic_dec_and_test(&ashmem_shrink_inflight)) + wake_up_all(&ashmem_shrink_wait); +} + /* * ashmem_shrink - our cache shrinker, called from mm/vmscan.c * @@ -449,12 +475,18 @@ ashmem_shrink_scan(struct shrinker *shrink, struct shrink_control *sc) return -1; list_for_each_entry_safe(range, next, &ashmem_lru_list, lru) { - loff_t start = range->pgstart * PAGE_SIZE; - loff_t end = (range->pgend + 1) * PAGE_SIZE; + struct ashmem_shrink_work *w = kzalloc(sizeof(*w), GFP_ATOMIC); + + if (!w) + break; + INIT_WORK(&w->work, ashmem_shrink_worker); + w->file = range->asma->file; + get_file(w->file); + w->start = range->pgstart * PAGE_SIZE; + w->end = (range->pgend + 1) * PAGE_SIZE; + atomic_inc(&ashmem_shrink_inflight); + queue_work(ashmem_wq, &w->work); - range->asma->file->f_op->fallocate(range->asma->file, - FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE, - start, end - start); range->purged = ASHMEM_WAS_PURGED; lru_del(range); @@ -713,6 +745,7 @@ static int ashmem_pin_unpin(struct ashmem_area *asma, unsigned long cmd, return -EFAULT; mutex_lock(&ashmem_mutex); + wait_event(ashmem_shrink_wait, !atomic_read(&ashmem_shrink_inflight)); if (!asma->file) goto out_unlock; @@ -883,8 +916,15 @@ static int __init ashmem_init(void) goto out_free2; } + ashmem_wq = alloc_workqueue("ashmem_wq", WQ_MEM_RECLAIM, 0); + if (!ashmem_wq) { + pr_err("failed to create workqueue\n"); + goto out_demisc; + } + ret = register_shrinker(&ashmem_shrinker); if (ret) { + destroy_workqueue(ashmem_wq); pr_err("failed to register shrinker!\n"); goto out_demisc; } ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: possible deadlock in vfs_fallocate 2019-01-22 13:58 ` Dmitry Vyukov @ 2019-01-22 14:26 ` syzbot 0 siblings, 0 replies; 4+ messages in thread From: syzbot @ 2019-01-22 14:26 UTC (permalink / raw) To: arve, devel, dvyukov, ebiggers3, gregkh, joelaf, linux-fsdevel, linux-kernel, maco, penguin-kernel, syzkaller-bugs, tkjos, tkjos Hello, syzbot has tested the proposed patch and the reproducer did not trigger crash: Reported-and-tested-by: syzbot+148c2885d71194f18d28@syzkaller.appspotmail.com Tested on: commit: 48b161983ae5 Merge tag 'xarray-5.0-rc3' of git://git.infra.. git tree: upstream kernel config: https://syzkaller.appspot.com/x/.config?x=dd3a020d055fa2e8 compiler: gcc (GCC) 9.0.0 20181231 (experimental) patch: https://syzkaller.appspot.com/x/patch.diff?x=16e6d908c00000 Note: testing is done by a robot and is best-effort only. ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2019-01-22 14:26 UTC | newest] Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-04-29 17:00 possible deadlock in vfs_fallocate syzbot 2018-05-09 7:57 ` Eric Biggers [not found] ` <CAHRSSEwAjrxZLRa3tPhjvz2MFiqVbCysTTgp6oKc=3Pcr1b2JQ@mail.gmail.com> 2019-01-22 13:58 ` Dmitry Vyukov 2019-01-22 14:26 ` syzbot
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).