Re: INFO: task hung in vhost_net_stop_vq

From: Dmitry Vyukov <dvyukov@google.com>
To: Jason Wang <jasowang@redhat.com>
Cc: "Michael S. Tsirkin" <mst@redhat.com>,
	syzbot <syzbot+d21e6e297322a900c128@syzkaller.appspotmail.com>,
	KVM list <kvm@vger.kernel.org>,
	LKML <linux-kernel@vger.kernel.org>,
	netdev <netdev@vger.kernel.org>,
	syzkaller-bugs <syzkaller-bugs@googlegroups.com>,
	virtualization@lists.linux-foundation.org, weiyj.lk@gmail.com
Subject: Re: INFO: task hung in vhost_net_stop_vq
Date: Tue, 26 Mar 2019 11:28:18 +0100	[thread overview]
Message-ID: <CACT4Y+Z1s-Lx1UXHKj88kQoOcbiD8gwyuRU3F_+cceP3pzbbrw@mail.gmail.com> (raw)
In-Reply-To: <df4f2cf6-8469-f894-8f45-7c48a6a1801f@redhat.com>

On Tue, Mar 26, 2019 at 11:17 AM Jason Wang <jasowang@redhat.com> wrote:
>
>
> On 2019/3/25 下午10:02, Michael S. Tsirkin wrote:
> > Looks like more iotlb locking mess?
>
>
> Looking at the calltrace:
>
> [  221.743675] =============================================
> [  221.744297] [ INFO: possible recursive locking detected ]
> [  221.744944] 4.7.0+ #1 Not tainted
> [  221.745326] ---------------------------------------------
> [  221.746128] syz-executor1/6823 is trying to acquire lock:
> [  221.746737]  (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0
> [  221.747789]
> [  221.747789] but task is already holding lock:
> [  221.748470]  (&vq->mutex){+.+...}, at: [<ffffffff84484b70>] vhost_process_iotlb_msg+0xe0/0x9e0
> [  221.749535]
> [  221.749535] other info that might help us debug this:
> [  221.750280]  Possible unsafe locking scenario:
> [  221.750280]
> [  221.750946]        CPU0
> [  221.751232]        ----
> [  221.751523]   lock(&vq->mutex);
> [  221.751922]   lock(&vq->mutex);
> [  221.752339]
> [  221.752339]  *** DEADLOCK ***
> [  221.752339]
>
> I could not think of a path that can hit this. And I could not reproduce with the reproducer in the link in net-next.

Looking at the bisection log, syzbot is able to reproduce this
super-reliably on multiple kernel revisions. Are you sure you are
using the right config/revision? What else can be in play? syzbot uses
VMs. The image is available.

> Thanks
>
>
> >
> > On Tue, Mar 19, 2019 at 10:21:00PM -0700, syzbot wrote:
> >> syzbot has bisected this bug to:
> >>
> >> commit 6b1e6cc7855b09a0a9bfa1d9f30172ba366f161c
> >> Author: Jason Wang <jasowang@redhat.com>
> >> Date:   Thu Jun 23 06:04:32 2016 +0000
> >>
> >>      vhost: new device IOTLB API
> >>
> >> bisection log:  https://syzkaller.appspot.com/x/bisect.txt?x=1486ad27200000
> >> start commit:   6b1e6cc7 vhost: new device IOTLB API
> >> git tree:       upstream
> >> final crash:    https://syzkaller.appspot.com/x/report.txt?x=1686ad27200000
> >> console output: https://syzkaller.appspot.com/x/log.txt?x=1286ad27200000
> >> kernel config:  https://syzkaller.appspot.com/x/.config?x=c94f9f0c0363db4b
> >> dashboard link: https://syzkaller.appspot.com/bug?extid=d21e6e297322a900c128
> >> syz repro:      https://syzkaller.appspot.com/x/repro.syz?x=141db34d400000
> >> C reproducer:   https://syzkaller.appspot.com/x/repro.c?x=108ef293400000
> >>
> >> Reported-by: syzbot+d21e6e297322a900c128@syzkaller.appspotmail.com
> >> Fixes: 6b1e6cc7 ("vhost: new device IOTLB API")
>
> --
> You received this message because you are subscribed to the Google Groups "syzkaller-bugs" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to syzkaller-bugs+unsubscribe@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/syzkaller-bugs/df4f2cf6-8469-f894-8f45-7c48a6a1801f%40redhat.com.
> For more options, visit https://groups.google.com/d/optout.