RE: possible deadlock in cma_netdev_callback

["Bernard Metzler" <BMT@zurich.ibm.com>]

-----"Jason Gunthorpe" <jgg@ziepe.ca> wrote: -----

>To: "syzbot" <syzbot+55de90ab5f44172b0c90@syzkaller.appspotmail.com>
>From: "Jason Gunthorpe" <jgg@ziepe.ca>
>Date: 02/26/2020 09:42PM
>Cc: chuck.lever@oracle.com, dledford@redhat.com, leon@kernel.org,
>linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org,
>netdev@vger.kernel.org, parav@mellanox.com,
>syzkaller-bugs@googlegroups.com, willy@infradead.org, "Bernard
>Metzler" <bmt@zurich.ibm.com>
>Subject: [EXTERNAL] Re: possible deadlock in cma_netdev_callback
>
>On Tue, Feb 25, 2020 at 09:39:10PM -0800, syzbot wrote:
>> Hello,
>> 
>> syzbot found the following crash on:
>> 
>> HEAD commit:    6132c1d9 net: core: devlink.c: Hold devlink->lock
>from the..
>> git tree:       net
>> console output:
>https://urldefense.proofpoint.com/v2/url?u=https-3A__syzkaller.appspo
>t.com_x_log.txt-3Fx-3D16978909e00000&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZO
>g&r=2TaYXQ0T-r8ZO1PP1alNwU_QJcRRLfmYTAgd3QCvqSc&m=I4mBjC4dKAL61lpQH5f
>iT4hLQEibtRif2HwliI2VpTA&s=Pd7_6w9kZzU3DupxBL6qo6piAhk8us2gO-BbCVTDj3
>Q&e= 
>> kernel config:
>https://urldefense.proofpoint.com/v2/url?u=https-3A__syzkaller.appspo
>t.com_x_.config-3Fx-3D3b8906eb6a7d6028&d=DwIBAg&c=jf_iaSHvJObTbx-siA1
>ZOg&r=2TaYXQ0T-r8ZO1PP1alNwU_QJcRRLfmYTAgd3QCvqSc&m=I4mBjC4dKAL61lpQH
>5fiT4hLQEibtRif2HwliI2VpTA&s=qI_ppGZR3Vy01oD9xwfU3or7fBrclf20NYgmTJ0N
>v4k&e= 
>> dashboard link:
>https://urldefense.proofpoint.com/v2/url?u=https-3A__syzkaller.appspo
>t.com_bug-3Fextid-3D55de90ab5f44172b0c90&d=DwIBAg&c=jf_iaSHvJObTbx-si
>A1ZOg&r=2TaYXQ0T-r8ZO1PP1alNwU_QJcRRLfmYTAgd3QCvqSc&m=I4mBjC4dKAL61lp
>QH5fiT4hLQEibtRif2HwliI2VpTA&s=OCNawVVe2X3ySwQUmRx_s2XM3p0r4d4cMFkYU_
>IIAmM&e= 
>> compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
>> syz repro:
>https://urldefense.proofpoint.com/v2/url?u=https-3A__syzkaller.appspo
>t.com_x_repro.syz-3Fx-3D12808281e00000&d=DwIBAg&c=jf_iaSHvJObTbx-siA1
>ZOg&r=2TaYXQ0T-r8ZO1PP1alNwU_QJcRRLfmYTAgd3QCvqSc&m=I4mBjC4dKAL61lpQH
>5fiT4hLQEibtRif2HwliI2VpTA&s=_-Ba4z4VxFdS5ran1HOTqcCl5KtbdPUvvthP_yOT
>bJw&e= 
>> C reproducer:
>https://urldefense.proofpoint.com/v2/url?u=https-3A__syzkaller.appspo
>t.com_x_repro.c-3Fx-3D134ca6fde00000&d=DwIBAg&c=jf_iaSHvJObTbx-siA1ZO
>g&r=2TaYXQ0T-r8ZO1PP1alNwU_QJcRRLfmYTAgd3QCvqSc&m=I4mBjC4dKAL61lpQH5f
>iT4hLQEibtRif2HwliI2VpTA&s=tTrtxQFoWaR89fJY7q7Z2shNBhVzrshezgxE17uS34
>o&e= 
>> 
>> IMPORTANT: if you fix the bug, please add the following tag to the
>commit:
>> Reported-by: syzbot+55de90ab5f44172b0c90@syzkaller.appspotmail.com
>> 
>> iwpm_register_pid: Unable to send a nlmsg (client = 2)
>> infiniband syz1: RDMA CMA: cma_listen_on_dev, error -98
>> netlink: 'syz-executor639': attribute type 1 has an invalid length.
>> 8021q: adding VLAN 0 to HW filter on device bond1
>> bond1: (slave gretap1): making interface the new active one
>> ======================================================
>> WARNING: possible circular locking dependency detected
>> 5.6.0-rc2-syzkaller #0 Not tainted
>> syz-executor639/9689 is trying to acquire lock:
>> ffffffff8a5d2a60 (lock#3){+.+.}, at: cma_netdev_callback+0xc6/0x380
>drivers/infiniband/core/cma.c:4605
>> 
>> but task is already holding lock:
>> ffffffff8a74da00 (rtnl_mutex){+.+.}, at: rtnl_lock
>net/core/rtnetlink.c:72 [inline]
>> ffffffff8a74da00 (rtnl_mutex){+.+.}, at:
>rtnetlink_rcv_msg+0x405/0xaf0 net/core/rtnetlink.c:5433
>>
>
>Bernard, this is a siw bug too, it is not allowed to get RTNL in
>siw_create_listen() (though this is probably for silly reasons and
>could be fixed)
>
>It is not easy to get this into the lockdep, I'll send a different
>patch too
>
>Jason
Hi Jason,

Thanks for letting me know! Hmm, we cannot use RCU locks since
we potentially sleep. One solution would be to create a list
of matching interfaces while under lock, unlock and use that
list for calling siw_listen_address() (which may sleep),
right...?

Many thanks,
Bernard.