From: ebiederm@xmission.com (Eric W. Biederman)
To: Waiman Long <longman@redhat.com>
Cc: syzbot <syzbot+a9fb1457d720a55d6dc5@syzkaller.appspotmail.com>,
adobriyan@gmail.com, akpm@linux-foundation.org,
allison@lohutok.net, areber@redhat.com,
aubrey.li@linux.intel.com, avagin@gmail.com,
bfields@fieldses.org, christian@brauner.io, cyphar@cyphar.com,
gregkh@linuxfoundation.org, guro@fb.com, jlayton@kernel.org,
joel@joelfernandes.org, keescook@chromium.org,
linmiaohe@huawei.com, linux-fsdevel@vger.kernel.org,
linux-kernel@vger.kernel.org, mhocko@suse.com, mingo@kernel.org,
oleg@redhat.com, peterz@infradead.org, sargun@sargun.me,
syzkaller-bugs@googlegroups.com, tglx@linutronix.de,
viro@zeniv.linux.org.uk
Subject: Re: possible deadlock in send_sigio
Date: Thu, 11 Jun 2020 11:07:27 -0500 [thread overview]
Message-ID: <87pna5si0w.fsf@x220.int.ebiederm.org> (raw)
In-Reply-To: <69818a6c-7025-8950-da4b-7fdc065d90d6@redhat.com> (Waiman Long's message of "Wed, 10 Jun 2020 22:32:44 -0400")
Waiman Long <longman@redhat.com> writes:
> On 4/4/20 1:55 AM, syzbot wrote:
>> Hello,
>>
>> syzbot found the following crash on:
>>
>> HEAD commit: bef7b2a7 Merge tag 'devicetree-for-5.7' of git://git.kerne..
>> git tree: upstream
>> console output: https://syzkaller.appspot.com/x/log.txt?x=15f39c5de00000
>> kernel config: https://syzkaller.appspot.com/x/.config?x=91b674b8f0368e69
>> dashboard link: https://syzkaller.appspot.com/bug?extid=a9fb1457d720a55d6dc5
>> compiler: gcc (GCC) 9.0.0 20181231 (experimental)
>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=1454c3b7e00000
>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=12a22ac7e00000
>>
>> The bug was bisected to:
>>
>> commit 7bc3e6e55acf065500a24621f3b313e7e5998acf
>> Author: Eric W. Biederman <ebiederm@xmission.com>
>> Date: Thu Feb 20 00:22:26 2020 +0000
>>
>> proc: Use a list of inodes to flush from proc
>>
>> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=165c4acde00000
>> final crash: https://syzkaller.appspot.com/x/report.txt?x=155c4acde00000
>> console output: https://syzkaller.appspot.com/x/log.txt?x=115c4acde00000
>>
>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>> Reported-by: syzbot+a9fb1457d720a55d6dc5@syzkaller.appspotmail.com
>> Fixes: 7bc3e6e55acf ("proc: Use a list of inodes to flush from proc")
>>
>> ========================================================
>> WARNING: possible irq lock inversion dependency detected
>> 5.6.0-syzkaller #0 Not tainted
>> --------------------------------------------------------
>> ksoftirqd/0/9 just changed the state of lock:
>> ffffffff898090d8 (tasklist_lock){.+.?}-{2:2}, at: send_sigio+0xa9/0x340 fs/fcntl.c:800
>> but this lock took another, SOFTIRQ-unsafe lock in the past:
>> (&pid->wait_pidfd){+.+.}-{2:2}
>>
>>
>> and interrupts could create inverse lock ordering between them.
>>
>>
>> other info that might help us debug this:
>> Possible interrupt unsafe locking scenario:
>>
>> CPU0 CPU1
>> ---- ----
>> lock(&pid->wait_pidfd);
>> local_irq_disable();
>> lock(tasklist_lock);
>> lock(&pid->wait_pidfd);
>> <Interrupt>
>> lock(tasklist_lock);
>>
>> *** DEADLOCK ***
>
> That is a false positive. The qrwlock has the special property that it becomes
> unfair (for read lock) at interrupt context. So unless it is taking a write lock
> in the interrupt context, it won't go into deadlock. The current lockdep code
> does not capture the full semantics of qrwlock leading to this false positive.
>
Whatever it was it was fixed with:
63f818f46af9 ("proc: Use a dedicated lock in struct pid")
It is classic lock inversion caused by not disabling irqs.
Unless I am completely mistaken any non-irq code path that does:
write_lock_irq(&tasklist_lock);
spin_lock(&pid->lock);
Is susceptible to deadlock with:
spin_lock(&pid->lock);
<Interrupt>
read_lock(&task_list_lock);
Because it remains a lock inversion even with only a read lock taken in
irq context in irq context.
Eric
prev parent reply other threads:[~2020-06-11 16:11 UTC|newest]
Thread overview: 16+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-04-04 5:55 possible deadlock in send_sigio syzbot
2020-06-11 2:32 ` Waiman Long
2020-06-11 7:43 ` Dmitry Vyukov
2020-06-11 13:51 ` Waiman Long
2020-06-11 14:22 ` Peter Zijlstra
2020-06-11 16:09 ` Waiman Long
2020-06-11 23:55 ` Boqun Feng
2020-06-12 1:55 ` Waiman Long
2020-06-12 7:01 ` Boqun Feng
2020-06-15 16:37 ` Waiman Long
2020-06-15 16:49 ` Matthew Wilcox
2020-06-15 17:13 ` Waiman Long
2020-06-15 20:40 ` Matthew Wilcox
2020-06-16 0:13 ` Boqun Feng
2020-06-16 0:31 ` Waiman Long
2020-06-11 16:07 ` Eric W. Biederman [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=87pna5si0w.fsf@x220.int.ebiederm.org \
--to=ebiederm@xmission.com \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=allison@lohutok.net \
--cc=areber@redhat.com \
--cc=aubrey.li@linux.intel.com \
--cc=avagin@gmail.com \
--cc=bfields@fieldses.org \
--cc=christian@brauner.io \
--cc=cyphar@cyphar.com \
--cc=gregkh@linuxfoundation.org \
--cc=guro@fb.com \
--cc=jlayton@kernel.org \
--cc=joel@joelfernandes.org \
--cc=keescook@chromium.org \
--cc=linmiaohe@huawei.com \
--cc=linux-fsdevel@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=longman@redhat.com \
--cc=mhocko@suse.com \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=sargun@sargun.me \
--cc=syzbot+a9fb1457d720a55d6dc5@syzkaller.appspotmail.com \
--cc=syzkaller-bugs@googlegroups.com \
--cc=tglx@linutronix.de \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).