From: Dmitry Vyukov <dvyukov@google.com>
To: Eric Biggers <ebiggers3@gmail.com>
Cc: syzbot
<bot+e93a80c1bb7c5c56e522461c149f8bf55eab1b2b@syzkaller.appspotmail.com>,
linux-fsdevel@vger.kernel.org,
LKML <linux-kernel@vger.kernel.org>,
syzkaller-bugs@googlegroups.com,
Al Viro <viro@zeniv.linux.org.uk>
Subject: Re: possible deadlock in seq_read
Date: Wed, 27 Dec 2017 19:30:39 +0100 [thread overview]
Message-ID: <CACT4Y+Y0+T+dmg4sfz_t_EZix1BeP87v+bPgjumqfNf573h3aQ@mail.gmail.com> (raw)
In-Reply-To: <20171212220647.GJ185376@gmail.com>
On Tue, Dec 12, 2017 at 11:06 PM, Eric Biggers <ebiggers3@gmail.com> wrote:
> On Fri, Dec 01, 2017 at 03:29:01AM -0800, syzbot wrote:
>> Hello,
>>
>> syzkaller hit the following crash on
>> df8ba95c572a187ed2aa7403e97a7a7f58c01f00
>> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master
>> compiler: gcc (GCC) 7.1.1 20170620
>> .config is attached
>> Raw console output is attached.
>>
>> Unfortunately, I don't have any reproducer for this bug yet.
>>
>>
>>
>> ======================================================
>> WARNING: possible circular locking dependency detected
>> 4.15.0-rc1+ #202 Not tainted
>> ------------------------------------------------------
>> syz-executor4/26476 is trying to acquire lock:
>> (&p->lock){+.+.}, at: [<0000000040185b66>] seq_read+0xd5/0x13d0
>> fs/seq_file.c:165
>>
>> but task is already holding lock:
>> (&pipe->mutex/1){+.+.}, at: [<00000000c644bcdc>] pipe_lock_nested
>> fs/pipe.c:67 [inline]
>> (&pipe->mutex/1){+.+.}, at: [<00000000c644bcdc>]
>> pipe_lock+0x56/0x70 fs/pipe.c:75
>>
>> which lock already depends on the new lock.
>>
>>
>> the existing dependency chain (in reverse order) is:
>>
>> -> #2 (&pipe->mutex/1){+.+.}:
>> lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4004
>> __mutex_lock_common kernel/locking/mutex.c:756 [inline]
>> __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
>> mutex_lock_nested+0x16/0x20 kernel/locking/mutex.c:908
>> __pipe_lock fs/pipe.c:88 [inline]
>> fifo_open+0x15c/0xa40 fs/pipe.c:916
>> do_dentry_open+0x682/0xd70 fs/open.c:752
>> vfs_open+0x107/0x230 fs/open.c:866
>> do_last fs/namei.c:3379 [inline]
>> path_openat+0x1157/0x3530 fs/namei.c:3519
>> do_filp_open+0x25b/0x3b0 fs/namei.c:3554
>> do_open_execat+0x1b9/0x5c0 fs/exec.c:849
>> do_execveat_common.isra.30+0x90c/0x23c0 fs/exec.c:1741
>> do_execveat fs/exec.c:1859 [inline]
>> SYSC_execveat fs/exec.c:1940 [inline]
>> SyS_execveat+0x4f/0x60 fs/exec.c:1932
>> do_syscall_64+0x26c/0x920 arch/x86/entry/common.c:285
>> return_from_SYSCALL_64+0x0/0x75
>>
>> -> #1 (&sig->cred_guard_mutex){+.+.}:
>> lock_acquire+0x1d5/0x580 kernel/locking/lockdep.c:4004
>> __mutex_lock_common kernel/locking/mutex.c:756 [inline]
>> __mutex_lock+0x16f/0x1a80 kernel/locking/mutex.c:893
>> mutex_lock_killable_nested+0x16/0x20 kernel/locking/mutex.c:923
>> do_io_accounting+0x1c2/0xf50 fs/proc/base.c:2682
>> proc_tid_io_accounting+0x1f/0x30 fs/proc/base.c:2725
>> proc_single_show+0xf8/0x170 fs/proc/base.c:744
>> seq_read+0x385/0x13d0 fs/seq_file.c:234
>> __vfs_read+0xef/0xa00 fs/read_write.c:411
>> vfs_read+0x124/0x360 fs/read_write.c:447
>> SYSC_read fs/read_write.c:573 [inline]
>> SyS_read+0xef/0x220 fs/read_write.c:566
>> entry_SYSCALL_64_fastpath+0x1f/0x96
>>
>
> So the problem with all these deadlocks involving pipe->mutex and
> sig->cred_guard_mutex is that execve() ranks pipe->mutex below
> sig->cred_guard_mutex when it tries to open a fifo, whereas reading or writing
> some of the /proc files result in ->cred_guard_mutex being taken which may be
> underneath pipe->mutex from splice(). Here's a program which causes an actual
> deadlock using this bug (in addition to reproducing the lockdep report):
>
> #define _GNU_SOURCE
> #include <fcntl.h>
> #include <pthread.h>
> #include <sys/stat.h>
> #include <unistd.h>
>
> static void *exec_thread(void *_arg)
> {
> for (;;)
> execl("fifo", "fifo", NULL);
> }
>
> int main()
> {
> int readend, writeend;
> int syscallfd;
> pthread_t t;
>
> mknod("fifo", 0777|S_IFIFO, 0);
> readend = open("fifo", O_RDONLY|O_NONBLOCK);
> writeend = open("fifo", O_WRONLY);
> syscallfd = open("/proc/self/syscall", O_RDONLY);
>
> pthread_create(&t, NULL, exec_thread, NULL);
>
> for (;;) {
> char buffer[16];
> loff_t off_in = 0;
> splice(syscallfd, &off_in, writeend, NULL, 16, 0);
> read(readend, buffer, 16);
> }
> }
>
> I'm not sure what the fix will be. Maybe the proc handlers should take a
> different lock instead of cred_guard_mutex. Or perhaps execve should check that
> the file is a regular file before it attempts to open it.
up
still happens and this is one of top crashers with 20K crashes
next prev parent reply other threads:[~2017-12-27 18:31 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <001a113f711ae2110c055f45acb8@google.com>
2017-12-02 8:12 ` possible deadlock in seq_read syzbot
2017-12-03 22:02 ` syzbot
2017-12-12 22:06 ` Eric Biggers
2017-12-27 18:30 ` Dmitry Vyukov [this message]
2018-08-27 18:20 ` Kees Cook
2018-08-27 20:57 ` Dmitry Vyukov
2019-01-21 10:14 ` [PATCH] fs: Allow opening only regular files during execve() Tetsuo Handa
2019-01-21 20:24 ` Kees Cook
2019-01-21 21:18 ` Al Viro
2019-01-22 0:50 ` [PATCH v2] " Tetsuo Handa
2019-01-28 23:42 ` Andrew Morton
2019-02-12 2:01 ` Tetsuo Handa
2019-02-12 5:26 ` Tetsuo Handa
2019-02-19 9:51 ` Tetsuo Handa
2019-01-22 0:51 ` [PATCH] " Kees Cook
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CACT4Y+Y0+T+dmg4sfz_t_EZix1BeP87v+bPgjumqfNf573h3aQ@mail.gmail.com \
--to=dvyukov@google.com \
--cc=bot+e93a80c1bb7c5c56e522461c149f8bf55eab1b2b@syzkaller.appspotmail.com \
--cc=ebiggers3@gmail.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=syzkaller-bugs@googlegroups.com \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).