linux-fsdevel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: ebiederm@xmission.com (Eric W. Biederman)
To: Waiman Long <longman@redhat.com>
Cc: syzbot <syzbot+a9fb1457d720a55d6dc5@syzkaller.appspotmail.com>,
	adobriyan@gmail.com, akpm@linux-foundation.org,
	allison@lohutok.net, areber@redhat.com,
	aubrey.li@linux.intel.com, avagin@gmail.com,
	bfields@fieldses.org, christian@brauner.io, cyphar@cyphar.com,
	gregkh@linuxfoundation.org, guro@fb.com, jlayton@kernel.org,
	joel@joelfernandes.org, keescook@chromium.org,
	linmiaohe@huawei.com, linux-fsdevel@vger.kernel.org,
	linux-kernel@vger.kernel.org, mhocko@suse.com, mingo@kernel.org,
	oleg@redhat.com, peterz@infradead.org, sargun@sargun.me,
	syzkaller-bugs@googlegroups.com, tglx@linutronix.de,
	viro@zeniv.linux.org.uk
Subject: Re: possible deadlock in send_sigio
Date: Thu, 11 Jun 2020 11:07:27 -0500	[thread overview]
Message-ID: <87pna5si0w.fsf@x220.int.ebiederm.org> (raw)
In-Reply-To: <69818a6c-7025-8950-da4b-7fdc065d90d6@redhat.com> (Waiman Long's message of "Wed, 10 Jun 2020 22:32:44 -0400")

Waiman Long <longman@redhat.com> writes:

> On 4/4/20 1:55 AM, syzbot wrote:
>> Hello,
>>
>> syzbot found the following crash on:
>>
>> HEAD commit:    bef7b2a7 Merge tag 'devicetree-for-5.7' of git://git.kerne..
>> git tree:       upstream
>> console output: https://syzkaller.appspot.com/x/log.txt?x=15f39c5de00000
>> kernel config:  https://syzkaller.appspot.com/x/.config?x=91b674b8f0368e69
>> dashboard link: https://syzkaller.appspot.com/bug?extid=a9fb1457d720a55d6dc5
>> compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
>> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=1454c3b7e00000
>> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=12a22ac7e00000
>>
>> The bug was bisected to:
>>
>> commit 7bc3e6e55acf065500a24621f3b313e7e5998acf
>> Author: Eric W. Biederman <ebiederm@xmission.com>
>> Date:   Thu Feb 20 00:22:26 2020 +0000
>>
>>      proc: Use a list of inodes to flush from proc
>>
>> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=165c4acde00000
>> final crash:    https://syzkaller.appspot.com/x/report.txt?x=155c4acde00000
>> console output: https://syzkaller.appspot.com/x/log.txt?x=115c4acde00000
>>
>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
>> Reported-by: syzbot+a9fb1457d720a55d6dc5@syzkaller.appspotmail.com
>> Fixes: 7bc3e6e55acf ("proc: Use a list of inodes to flush from proc")
>>
>> ========================================================
>> WARNING: possible irq lock inversion dependency detected
>> 5.6.0-syzkaller #0 Not tainted
>> --------------------------------------------------------
>> ksoftirqd/0/9 just changed the state of lock:
>> ffffffff898090d8 (tasklist_lock){.+.?}-{2:2}, at: send_sigio+0xa9/0x340 fs/fcntl.c:800
>> but this lock took another, SOFTIRQ-unsafe lock in the past:
>>   (&pid->wait_pidfd){+.+.}-{2:2}
>>
>>
>> and interrupts could create inverse lock ordering between them.
>>
>>
>> other info that might help us debug this:
>>   Possible interrupt unsafe locking scenario:
>>
>>         CPU0                    CPU1
>>         ----                    ----
>>    lock(&pid->wait_pidfd);
>>                                 local_irq_disable();
>>                                 lock(tasklist_lock);
>>                                 lock(&pid->wait_pidfd);
>>    <Interrupt>
>>      lock(tasklist_lock);
>>
>>   *** DEADLOCK ***
>
> That is a false positive. The qrwlock has the special property that it becomes
> unfair (for read lock) at interrupt context. So unless it is taking a write lock
> in the interrupt context, it won't go into deadlock. The current lockdep code
> does not capture the full semantics of qrwlock leading to this false positive.
>

Whatever it was it was fixed with:
63f818f46af9 ("proc: Use a dedicated lock in struct pid")

It is classic lock inversion caused by not disabling irqs.

Unless I am completely mistaken any non-irq code path that does:
	write_lock_irq(&tasklist_lock);
        spin_lock(&pid->lock);

Is susceptible to deadlock with:
	spin_lock(&pid->lock);
        <Interrupt>
        read_lock(&task_list_lock);

Because it remains a lock inversion even with only a read lock taken in
irq context in irq context.

Eric

      parent reply	other threads:[~2020-06-11 16:11 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-04-04  5:55 possible deadlock in send_sigio syzbot
2020-06-11  2:32 ` Waiman Long
2020-06-11  7:43   ` Dmitry Vyukov
2020-06-11 13:51     ` Waiman Long
2020-06-11 14:22       ` Peter Zijlstra
2020-06-11 16:09         ` Waiman Long
2020-06-11 23:55           ` Boqun Feng
2020-06-12  1:55             ` Waiman Long
2020-06-12  7:01             ` Boqun Feng
2020-06-15 16:37               ` Waiman Long
2020-06-15 16:49               ` Matthew Wilcox
2020-06-15 17:13                 ` Waiman Long
2020-06-15 20:40                   ` Matthew Wilcox
2020-06-16  0:13                     ` Boqun Feng
2020-06-16  0:31                       ` Waiman Long
2020-06-11 16:07   ` Eric W. Biederman [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=87pna5si0w.fsf@x220.int.ebiederm.org \
    --to=ebiederm@xmission.com \
    --cc=adobriyan@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=allison@lohutok.net \
    --cc=areber@redhat.com \
    --cc=aubrey.li@linux.intel.com \
    --cc=avagin@gmail.com \
    --cc=bfields@fieldses.org \
    --cc=christian@brauner.io \
    --cc=cyphar@cyphar.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=guro@fb.com \
    --cc=jlayton@kernel.org \
    --cc=joel@joelfernandes.org \
    --cc=keescook@chromium.org \
    --cc=linmiaohe@huawei.com \
    --cc=linux-fsdevel@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longman@redhat.com \
    --cc=mhocko@suse.com \
    --cc=mingo@kernel.org \
    --cc=oleg@redhat.com \
    --cc=peterz@infradead.org \
    --cc=sargun@sargun.me \
    --cc=syzbot+a9fb1457d720a55d6dc5@syzkaller.appspotmail.com \
    --cc=syzkaller-bugs@googlegroups.com \
    --cc=tglx@linutronix.de \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).