* excuse-lockdep help
[not found] ` <16905BAB-7AB6-4170-98CC-292F0C01F30F@oracle.com>
@ 2021-04-06 1:31 ` jun.miao
2021-04-06 3:13 ` Trond Myklebust
0 siblings, 1 reply; 2+ messages in thread
From: jun.miao @ 2021-04-06 1:31 UTC (permalink / raw)
To: linux-nfs; +Cc: anna.schumaker, trond.myklebust
Hi maintainer,and stranger v5.4-rt kernel, there some BUGs error:
[ 29.286213] BUG: workqueue leaked lock or atomic: kworker/u8:0/0x00000000/7
last function: rpc_async_schedule
[ 29.286215] 9 locks held by kworker/u8:0/7:
[ 29.286217] #0: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286222] #1: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286226] #2: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286230] #3: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286241] #4: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286245] #5: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286249] #6: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286253] #7: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286258] #8: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
[ 29.286262] CPU: 2 PID: 7 Comm: kworker/u8:0 Tainted: G W 5.4.107-rt44-yocto-preempt-rt #1
[ 29.286263] Hardware name: Intel Corporation NUC7i5DNKE/NUC7i5DNB, BIOS DNKBLi5v.86A.0026.2017.0915.2016 09/15/2017
[ 29.286266] Workqueue: rpciod rpc_async_schedule
[ 29.286266] Call Trace:
[ 29.286268] show_stack+0x4e/0x52
[ 29.286272] dump_stack+0x7a/0xa4
[ 29.286276] process_one_work.cold+0x43/0x48
[ 29.286292] worker_thread+0x217/0x330
[ 29.286300] kthread+0x177/0x190
[ 29.286302] ? process_one_work+0x410/0x410
[ 29.286303] ? __kthread_parkme+0xc0/0xc0
[ 29.286310] ret_from_fork+0x3a/0x50
On 4/6/21 1:57 AM, Chuck Lever III wrote:
> [Please note: This e-mail is from an EXTERNAL e-mail address]
>
>> On Mar 28, 2021, at 10:31 PM, jun.miao <jun.miao@windriver.com> wrote:
>>
>> Hi maintainers
>>
>> I find a BUG calltrace in v5.4-rt:
>>
>> [ 29.286213] BUG: workqueue leaked lock or atomic: kworker/u8:0/0x00000000/7
>> last function: rpc_async_schedule
>> [ 29.286215] 9 locks held by kworker/u8:0/7:
>> [ 29.286217] #0: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286222] #1: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286226] #2: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286230] #3: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286241] #4: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286245] #5: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286249] #6: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286253] #7: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286258] #8: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 29.286262] CPU: 2 PID: 7 Comm: kworker/u8:0 Tainted: G W 5.4.107-rt44-yocto-preempt-rt #1
>> [ 29.286263] Hardware name: Intel Corporation NUC7i5DNKE/NUC7i5DNB, BIOS DNKBLi5v.86A.0026.2017.0915.2016 09/15/2017
>> [ 29.286266] Workqueue: rpciod rpc_async_schedule
>> [ 29.286266] Call Trace:
>> [ 29.286268] show_stack+0x4e/0x52
>> [ 29.286272] dump_stack+0x7a/0xa4
>> [ 29.286276] process_one_work.cold+0x43/0x48
>> [ 29.286292] worker_thread+0x217/0x330
>> [ 29.286300] kthread+0x177/0x190
>> [ 29.286302] ? process_one_work+0x410/0x410
>> [ 29.286303] ? __kthread_parkme+0xc0/0xc0
>> [ 29.286310] ret_from_fork+0x3a/0x50
>>
>>
>> When i git am this patch:
>>
>> commit 1a33e10e4a95cb109ff1145098175df3113313ef
>> Author: Cong Wang <xiyou.wangcong@gmail.com>
>> Date: Sat May 2 22:22:19 2020 -0700
>>
>> net: partially revert dynamic lockdep key changes
>>
>> The BUG also print:
>>
>> BUG: workqueue leaked lock or atomic: kworker/u8:0/0x00000000/7
>> [ 28.869159] 000: last function: rpc_async_release
>> [ 28.869161] 000: 13 locks held by kworker/u8:0/7:
>> [ 28.869162] 000: #0: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869167] 000: #1: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869172] 000: #2: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869177] 000: #3: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869189] 000: #4: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869193] 000: #5: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869198] 000: #6: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869203] 000: #7: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869208] 000: #8: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869213] 000: #9: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869218] 000: #10: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869223] 000: #11: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869228] 000: #12: ffff90b817de42f0 (&(&sch->running)->seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
>> [ 28.869233] 000: CPU: 0 PID: 7 Comm: kworker/u8:0 Tainted: G W 5.4.107-rt44-yocto-preempt-rt #1
>> [ 28.869234] 000: Hardware name: Intel Corporation NUC7i5DNKE/NUC7i5DNB, BIOS DNKBLi5v.86A.0026.2017.0915.2016 09/15/2017
>> [ 28.869236] 000: Workqueue: nfsiod rpc_async_release
>> [ 28.869236] 000: Call Trace:
>> [ 28.869238] 000: show_stack+0x4e/0x52
>> [ 28.869242] 000: dump_stack+0x7a/0xa4
>> [ 28.869247] 000: process_one_work.cold+0x43/0x48
>> [ 28.869266] 000: worker_thread+0x53/0x330
>> [ 28.869275] 000: kthread+0x177/0x190
>> [ 28.869278] 000: ? process_one_work+0x410/0x410
>> [ 28.869279] 000: ? __kthread_parkme+0xc0/0xc0
>> [ 28.869286] 000: ret_from_fork+0x3a/0x50
>>
>>
>>
>> Could you give me some advice about rpc_async_schedule() in sunrpc File ?
>>
>> Thanks very much
> Hello-
>
> rpc_async_schedule() is part of the RPC client, whose maintainers
> are Trond Myklebust and Anna Schumaker. Please post this WQ splat
> to linux-nfs@vger.kernel.org and Cc: Trond and Anna. Thanks!
>
>
> --
> Chuck Lever
>
>
>
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: excuse-lockdep help
2021-04-06 1:31 ` excuse-lockdep help jun.miao
@ 2021-04-06 3:13 ` Trond Myklebust
0 siblings, 0 replies; 2+ messages in thread
From: Trond Myklebust @ 2021-04-06 3:13 UTC (permalink / raw)
To: linux-nfs, jun.miao; +Cc: anna.schumaker
On Tue, 2021-04-06 at 09:31 +0800, jun.miao wrote:
> Hi maintainer,and stranger v5.4-rt kernel, there some BUGs error:
>
> [ 29.286213] BUG: workqueue leaked lock or atomic:
> kworker/u8:0/0x00000000/7
> last function: rpc_async_schedule
> [ 29.286215] 9 locks held by kworker/u8:0/7:
> [ 29.286217] #0: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286222] #1: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286226] #2: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286230] #3: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286241] #4: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286245] #5: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286249] #6: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286253] #7: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286258] #8: ffff953b5745c2f0 (&dev->qdisc_running_key){+.+.},
> at: __dev_queue_xmit+0x32e/0x610
> [ 29.286262] CPU: 2 PID: 7 Comm: kworker/u8:0 Tainted: G W
> 5.4.107-rt44-yocto-preempt-rt #1
> [ 29.286263] Hardware name: Intel Corporation NUC7i5DNKE/NUC7i5DNB,
> BIOS DNKBLi5v.86A.0026.2017.0915.2016 09/15/2017
> [ 29.286266] Workqueue: rpciod rpc_async_schedule
> [ 29.286266] Call Trace:
> [ 29.286268] show_stack+0x4e/0x52
> [ 29.286272] dump_stack+0x7a/0xa4
> [ 29.286276] process_one_work.cold+0x43/0x48
> [ 29.286292] worker_thread+0x217/0x330
> [ 29.286300] kthread+0x177/0x190
> [ 29.286302] ? process_one_work+0x410/0x410
> [ 29.286303] ? __kthread_parkme+0xc0/0xc0
> [ 29.286310] ret_from_fork+0x3a/0x50
>
I can't see how this could be a RPC or NFS bug. The lock being leaked
is deep in the networking code, and is never taken by the higher
levels.
Furthermore, this is not an upstream kernel, but some Yocto project
franken-kernel with proprietary drivers having been loaded. I can't see
how we can support it.
>
>
> On 4/6/21 1:57 AM, Chuck Lever III wrote:
> > [Please note: This e-mail is from an EXTERNAL e-mail address]
> >
> > > On Mar 28, 2021, at 10:31 PM, jun.miao <jun.miao@windriver.com>
> > > wrote:
> > >
> > > Hi maintainers
> > >
> > > I find a BUG calltrace in v5.4-rt:
> > >
> > > [ 29.286213] BUG: workqueue leaked lock or atomic:
> > > kworker/u8:0/0x00000000/7
> > > last function: rpc_async_schedule
> > > [ 29.286215] 9 locks held by kworker/u8:0/7:
> > > [ 29.286217] #0: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286222] #1: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286226] #2: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286230] #3: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286241] #4: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286245] #5: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286249] #6: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286253] #7: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286258] #8: ffff953b5745c2f0 (&dev-
> > > >qdisc_running_key){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 29.286262] CPU: 2 PID: 7 Comm: kworker/u8:0 Tainted: G
> > > W 5.4.107-rt44-yocto-preempt-rt #1
> > > [ 29.286263] Hardware name: Intel Corporation
> > > NUC7i5DNKE/NUC7i5DNB, BIOS DNKBLi5v.86A.0026.2017.0915.2016
> > > 09/15/2017
> > > [ 29.286266] Workqueue: rpciod rpc_async_schedule
> > > [ 29.286266] Call Trace:
> > > [ 29.286268] show_stack+0x4e/0x52
> > > [ 29.286272] dump_stack+0x7a/0xa4
> > > [ 29.286276] process_one_work.cold+0x43/0x48
> > > [ 29.286292] worker_thread+0x217/0x330
> > > [ 29.286300] kthread+0x177/0x190
> > > [ 29.286302] ? process_one_work+0x410/0x410
> > > [ 29.286303] ? __kthread_parkme+0xc0/0xc0
> > > [ 29.286310] ret_from_fork+0x3a/0x50
> > >
> > >
> > > When i git am this patch:
> > >
> > > commit 1a33e10e4a95cb109ff1145098175df3113313ef
> > > Author: Cong Wang <xiyou.wangcong@gmail.com>
> > > Date: Sat May 2 22:22:19 2020 -0700
> > >
> > > net: partially revert dynamic lockdep key changes
> > >
> > > The BUG also print:
> > >
> > > BUG: workqueue leaked lock or atomic: kworker/u8:0/0x00000000/7
> > > [ 28.869159] 000: last function: rpc_async_release
> > > [ 28.869161] 000: 13 locks held by kworker/u8:0/7:
> > > [ 28.869162] 000: #0: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869167] 000: #1: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869172] 000: #2: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869177] 000: #3: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869189] 000: #4: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869193] 000: #5: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869198] 000: #6: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869203] 000: #7: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869208] 000: #8: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869213] 000: #9: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869218] 000: #10: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869223] 000: #11: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869228] 000: #12: ffff90b817de42f0 (&(&sch->running)-
> > > >seqcount){+.+.}, at: __dev_queue_xmit+0x32e/0x610
> > > [ 28.869233] 000: CPU: 0 PID: 7 Comm: kworker/u8:0 Tainted:
> > > G W 5.4.107-rt44-yocto-preempt-rt #1
> > > [ 28.869234] 000: Hardware name: Intel Corporation
> > > NUC7i5DNKE/NUC7i5DNB, BIOS DNKBLi5v.86A.0026.2017.0915.2016
> > > 09/15/2017
> > > [ 28.869236] 000: Workqueue: nfsiod rpc_async_release
> > > [ 28.869236] 000: Call Trace:
> > > [ 28.869238] 000: show_stack+0x4e/0x52
> > > [ 28.869242] 000: dump_stack+0x7a/0xa4
> > > [ 28.869247] 000: process_one_work.cold+0x43/0x48
> > > [ 28.869266] 000: worker_thread+0x53/0x330
> > > [ 28.869275] 000: kthread+0x177/0x190
> > > [ 28.869278] 000: ? process_one_work+0x410/0x410
> > > [ 28.869279] 000: ? __kthread_parkme+0xc0/0xc0
> > > [ 28.869286] 000: ret_from_fork+0x3a/0x50
> > >
> > >
> > >
> > > Could you give me some advice about rpc_async_schedule() in
> > > sunrpc File ?
> > >
> > > Thanks very much
> > Hello-
> >
> > rpc_async_schedule() is part of the RPC client, whose maintainers
> > are Trond Myklebust and Anna Schumaker. Please post this WQ splat
> > to linux-nfs@vger.kernel.org and Cc: Trond and Anna. Thanks!
> >
> >
> > --
> > Chuck Lever
> >
> >
> >
--
Trond Myklebust
Linux NFS client maintainer, Hammerspace
trond.myklebust@hammerspace.com
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2021-04-06 3:13 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <4dd50256-9b7b-73b2-29c7-1ca16cde73fc@windriver.com>
[not found] ` <a071d82f-d15f-e0f5-e483-9b7d1120f338@windriver.com>
[not found] ` <16905BAB-7AB6-4170-98CC-292F0C01F30F@oracle.com>
2021-04-06 1:31 ` excuse-lockdep help jun.miao
2021-04-06 3:13 ` Trond Myklebust
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.