* corrupted pvqspinlock in htab_map_update_elem
@ 2021-01-31 8:42 Dmitry Vyukov
2021-02-01 9:50 ` Peter Zijlstra
0 siblings, 1 reply; 8+ messages in thread
From: Dmitry Vyukov @ 2021-01-31 8:42 UTC (permalink / raw)
To: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau,
Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf,
LKML, Peter Zijlstra, Ingo Molnar
Hi,
I am testing the following the program:
https://gist.github.com/dvyukov/e5c0a8ef220ef856363c1080b0936a9e
on the latest upstream 6642d600b541b81931fb1ab0c041b0d68f77be7e and
getting the following crash. Config is:
https://gist.github.com/dvyukov/16d9905e5ef35e44285451f1d330ddbc
The program updates a bpf map from a program called on hw breakpoint
hit. Not sure if it's a bpf issue or a perf issue. This time it is not
a fuzzer workload, I am trying to do something useful :)
------------[ cut here ]------------
pvqspinlock: lock 0xffffffff8f371d80 has corrupted value 0x0!
WARNING: CPU: 3 PID: 8771 at kernel/locking/qspinlock_paravirt.h:498
__pv_queued_spin_unlock_slowpath+0x22e/0x2b0
kernel/locking/qspinlock_paravirt.h:498
Modules linked in:
CPU: 3 PID: 8771 Comm: a.out Not tainted 5.11.0-rc5+ #71
Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
rel-1.13.0-44-g88ab0c15525c-prebuilt.qemu.org 04/01/2014
RIP: 0010:__pv_queued_spin_unlock_slowpath+0x22e/0x2b0
kernel/locking/qspinlock_paravirt.h:498
Code: ea 03 0f b6 14 02 4c 89 e8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2
75 62 41 8b 55 00 4c 89 ee 48 c7 c7 20 6b 4c 89 e8 72 d3 5f 07 <0f> 0b
e9 6cc
RSP: 0018:fffffe00000c17b0 EFLAGS: 00010086
RAX: 0000000000000000 RBX: ffffffff8f3b5660 RCX: 0000000000000000
RDX: ffff8880150222c0 RSI: ffffffff815b624d RDI: fffffbc0000182e8
RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
R10: ffffffff817de94f R11: 0000000000000000 R12: ffff8880150222c0
R13: ffffffff8f371d80 R14: ffff8880181fead8 R15: 0000000000000000
FS: 00007fa5b51f0700(0000) GS:ffff88802cf80000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 0000000002286908 CR3: 0000000015b24000 CR4: 0000000000750ee0
DR0: 00000000004cb3d4 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
PKRU: 55555554
Call Trace:
<#DB>
__raw_callee_save___pv_queued_spin_unlock_slowpath+0x11/0x20
.slowpath+0x9/0xe
pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:559 [inline]
queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
print_usage_bug kernel/locking/lockdep.c:3710 [inline]
verify_lock_unused kernel/locking/lockdep.c:5374 [inline]
lock_acquire kernel/locking/lockdep.c:5433 [inline]
lock_acquire+0x471/0x720 kernel/locking/lockdep.c:5407
__raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
_raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
htab_lock_bucket kernel/bpf/hashtab.c:175 [inline]
htab_map_update_elem+0x1f0/0x790 kernel/bpf/hashtab.c:1023
bpf_prog_60236c52b8017ad1+0x8e/0xab4
bpf_dispatcher_nop_func include/linux/bpf.h:651 [inline]
bpf_overflow_handler+0x192/0x5b0 kernel/events/core.c:9755
__perf_event_overflow+0x13c/0x370 kernel/events/core.c:8979
perf_swevent_overflow kernel/events/core.c:9055 [inline]
perf_swevent_event+0x347/0x550 kernel/events/core.c:9083
perf_bp_event+0x1a2/0x1c0 kernel/events/core.c:9932
hw_breakpoint_handler arch/x86/kernel/hw_breakpoint.c:535 [inline]
hw_breakpoint_exceptions_notify+0x18a/0x3b0 arch/x86/kernel/hw_breakpoint.c:567
notifier_call_chain+0xb5/0x200 kernel/notifier.c:83
atomic_notifier_call_chain+0x8d/0x170 kernel/notifier.c:217
notify_die+0xda/0x170 kernel/notifier.c:548
notify_debug+0x20/0x30 arch/x86/kernel/traps.c:842
exc_debug_kernel arch/x86/kernel/traps.c:902 [inline]
exc_debug+0x103/0x140 arch/x86/kernel/traps.c:998
asm_exc_debug+0x19/0x30 arch/x86/include/asm/idtentry.h:598
RIP: 0010:copy_user_generic_unrolled+0xa2/0xc0 arch/x86/lib/copy_user_64.S:102
Code: ff c9 75 b6 89 d1 83 e2 07 c1 e9 03 74 12 4c 8b 06 4c 89 07 48
8d 76 08 48 8d 7f 08 ff c9 75 ee 21 d2 74 10 89 d1 8a 06 88 07 <48> ff
c6 484
RSP: 0018:ffffc90000d67af0 EFLAGS: 00040202
RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000001
RDX: 0000000000000001 RSI: ffff88801341d001 RDI: 00000000004cb3d4
RBP: 00000000004cb3d4 R08: 0000000000000000 R09: ffff88801341d001
R10: ffffed1002683a00 R11: 0000000000000000 R12: ffff88801341d001
R13: 00000000004cb3d5 R14: 0000000000000000 R15: 0000000000000001
</#DB>
copy_user_generic arch/x86/include/asm/uaccess_64.h:37 [inline]
raw_copy_to_user arch/x86/include/asm/uaccess_64.h:58 [inline]
copyout.part.0+0xe4/0x110 lib/iov_iter.c:148
copyout lib/iov_iter.c:182 [inline]
copy_page_to_iter_iovec lib/iov_iter.c:219 [inline]
copy_page_to_iter+0x416/0xed0 lib/iov_iter.c:926
pipe_read+0x4a4/0x13a0 fs/pipe.c:290
call_read_iter include/linux/fs.h:1895 [inline]
new_sync_read+0x5b7/0x6e0 fs/read_write.c:415
vfs_read+0x35c/0x570 fs/read_write.c:496
ksys_read+0x1ee/0x250 fs/read_write.c:634
do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46
entry_SYSCALL_64_after_hwframe+0x44/0xa9
RIP: 0033:0x406c1c
Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 f9 fc ff ff
48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d
00 f08
RSP: 002b:00007fa5b51f02d0 EFLAGS: 00000246 ORIG_RAX: 0000000000000000
RAX: ffffffffffffffda RBX: 00000000004cb3d4 RCX: 0000000000406c1c
RDX: 0000000000000001 RSI: 00000000004cb3d4 RDI: 0000000000000004
RBP: 00007fa5b51f030c R08: 0000000000000000 R09: 0000000000000000
R10: 00007fa5b51f0700 R11: 0000000000000246 R12: 00000000004cb3d0
R13: cccccccccccccccd R14: 00007fa5b51f0400 R15: 0000000000802000
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem
2021-01-31 8:42 corrupted pvqspinlock in htab_map_update_elem Dmitry Vyukov
@ 2021-02-01 9:50 ` Peter Zijlstra
2021-02-01 10:06 ` Dmitry Vyukov
2021-02-01 11:23 ` Peter Zijlstra
0 siblings, 2 replies; 8+ messages in thread
From: Peter Zijlstra @ 2021-02-01 9:50 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau,
Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf,
LKML, Ingo Molnar
On Sun, Jan 31, 2021 at 09:42:53AM +0100, Dmitry Vyukov wrote:
> Hi,
>
> I am testing the following the program:
> https://gist.github.com/dvyukov/e5c0a8ef220ef856363c1080b0936a9e
> on the latest upstream 6642d600b541b81931fb1ab0c041b0d68f77be7e and
> getting the following crash. Config is:
> https://gist.github.com/dvyukov/16d9905e5ef35e44285451f1d330ddbc
>
> The program updates a bpf map from a program called on hw breakpoint
> hit. Not sure if it's a bpf issue or a perf issue. This time it is not
> a fuzzer workload, I am trying to do something useful :)
Something useful and BPF don't go together as far as I'm concerned.
> ------------[ cut here ]------------
> pvqspinlock: lock 0xffffffff8f371d80 has corrupted value 0x0!
> WARNING: CPU: 3 PID: 8771 at kernel/locking/qspinlock_paravirt.h:498
> __pv_queued_spin_unlock_slowpath+0x22e/0x2b0
> kernel/locking/qspinlock_paravirt.h:498
> Modules linked in:
> CPU: 3 PID: 8771 Comm: a.out Not tainted 5.11.0-rc5+ #71
> Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
> rel-1.13.0-44-g88ab0c15525c-prebuilt.qemu.org 04/01/2014
> RIP: 0010:__pv_queued_spin_unlock_slowpath+0x22e/0x2b0
> kernel/locking/qspinlock_paravirt.h:498
> Code: ea 03 0f b6 14 02 4c 89 e8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2
> 75 62 41 8b 55 00 4c 89 ee 48 c7 c7 20 6b 4c 89 e8 72 d3 5f 07 <0f> 0b
> e9 6cc
> RSP: 0018:fffffe00000c17b0 EFLAGS: 00010086
> RAX: 0000000000000000 RBX: ffffffff8f3b5660 RCX: 0000000000000000
> RDX: ffff8880150222c0 RSI: ffffffff815b624d RDI: fffffbc0000182e8
> RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
> R10: ffffffff817de94f R11: 0000000000000000 R12: ffff8880150222c0
> R13: ffffffff8f371d80 R14: ffff8880181fead8 R15: 0000000000000000
> FS: 00007fa5b51f0700(0000) GS:ffff88802cf80000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 0000000002286908 CR3: 0000000015b24000 CR4: 0000000000750ee0
> DR0: 00000000004cb3d4 DR1: 0000000000000000 DR2: 0000000000000000
> DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> PKRU: 55555554
> Call Trace:
> <#DB>
> __raw_callee_save___pv_queued_spin_unlock_slowpath+0x11/0x20
> .slowpath+0x9/0xe
> pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:559 [inline]
> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
> print_usage_bug kernel/locking/lockdep.c:3710 [inline]
Ha, I think you hit a bug in lockdep. But it was about to tell you you
can't go take locks from NMI context that are also used outside of it.
> verify_lock_unused kernel/locking/lockdep.c:5374 [inline]
> lock_acquire kernel/locking/lockdep.c:5433 [inline]
> lock_acquire+0x471/0x720 kernel/locking/lockdep.c:5407
> __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
> _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
> htab_lock_bucket kernel/bpf/hashtab.c:175 [inline]
> htab_map_update_elem+0x1f0/0x790 kernel/bpf/hashtab.c:1023
> bpf_prog_60236c52b8017ad1+0x8e/0xab4
> bpf_dispatcher_nop_func include/linux/bpf.h:651 [inline]
> bpf_overflow_handler+0x192/0x5b0 kernel/events/core.c:9755
> __perf_event_overflow+0x13c/0x370 kernel/events/core.c:8979
> perf_swevent_overflow kernel/events/core.c:9055 [inline]
> perf_swevent_event+0x347/0x550 kernel/events/core.c:9083
> perf_bp_event+0x1a2/0x1c0 kernel/events/core.c:9932
> hw_breakpoint_handler arch/x86/kernel/hw_breakpoint.c:535 [inline]
> hw_breakpoint_exceptions_notify+0x18a/0x3b0 arch/x86/kernel/hw_breakpoint.c:567
> notifier_call_chain+0xb5/0x200 kernel/notifier.c:83
> atomic_notifier_call_chain+0x8d/0x170 kernel/notifier.c:217
> notify_die+0xda/0x170 kernel/notifier.c:548
> notify_debug+0x20/0x30 arch/x86/kernel/traps.c:842
> exc_debug_kernel arch/x86/kernel/traps.c:902 [inline]
> exc_debug+0x103/0x140 arch/x86/kernel/traps.c:998
> asm_exc_debug+0x19/0x30 arch/x86/include/asm/idtentry.h:598
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem
2021-02-01 9:50 ` Peter Zijlstra
@ 2021-02-01 10:06 ` Dmitry Vyukov
2021-02-01 11:23 ` Peter Zijlstra
1 sibling, 0 replies; 8+ messages in thread
From: Dmitry Vyukov @ 2021-02-01 10:06 UTC (permalink / raw)
To: Peter Zijlstra
Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau,
Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf,
LKML, Ingo Molnar
On Mon, Feb 1, 2021 at 10:51 AM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Sun, Jan 31, 2021 at 09:42:53AM +0100, Dmitry Vyukov wrote:
> > Hi,
> >
> > I am testing the following the program:
> > https://gist.github.com/dvyukov/e5c0a8ef220ef856363c1080b0936a9e
> > on the latest upstream 6642d600b541b81931fb1ab0c041b0d68f77be7e and
> > getting the following crash. Config is:
> > https://gist.github.com/dvyukov/16d9905e5ef35e44285451f1d330ddbc
> >
> > The program updates a bpf map from a program called on hw breakpoint
> > hit. Not sure if it's a bpf issue or a perf issue. This time it is not
> > a fuzzer workload, I am trying to do something useful :)
>
> Something useful and BPF don't go together as far as I'm concerned.
>
> > ------------[ cut here ]------------
> > pvqspinlock: lock 0xffffffff8f371d80 has corrupted value 0x0!
> > WARNING: CPU: 3 PID: 8771 at kernel/locking/qspinlock_paravirt.h:498
> > __pv_queued_spin_unlock_slowpath+0x22e/0x2b0
> > kernel/locking/qspinlock_paravirt.h:498
> > Modules linked in:
> > CPU: 3 PID: 8771 Comm: a.out Not tainted 5.11.0-rc5+ #71
> > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS
> > rel-1.13.0-44-g88ab0c15525c-prebuilt.qemu.org 04/01/2014
> > RIP: 0010:__pv_queued_spin_unlock_slowpath+0x22e/0x2b0
> > kernel/locking/qspinlock_paravirt.h:498
> > Code: ea 03 0f b6 14 02 4c 89 e8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2
> > 75 62 41 8b 55 00 4c 89 ee 48 c7 c7 20 6b 4c 89 e8 72 d3 5f 07 <0f> 0b
> > e9 6cc
> > RSP: 0018:fffffe00000c17b0 EFLAGS: 00010086
> > RAX: 0000000000000000 RBX: ffffffff8f3b5660 RCX: 0000000000000000
> > RDX: ffff8880150222c0 RSI: ffffffff815b624d RDI: fffffbc0000182e8
> > RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000
> > R10: ffffffff817de94f R11: 0000000000000000 R12: ffff8880150222c0
> > R13: ffffffff8f371d80 R14: ffff8880181fead8 R15: 0000000000000000
> > FS: 00007fa5b51f0700(0000) GS:ffff88802cf80000(0000) knlGS:0000000000000000
> > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> > CR2: 0000000002286908 CR3: 0000000015b24000 CR4: 0000000000750ee0
> > DR0: 00000000004cb3d4 DR1: 0000000000000000 DR2: 0000000000000000
> > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> > PKRU: 55555554
> > Call Trace:
> > <#DB>
> > __raw_callee_save___pv_queued_spin_unlock_slowpath+0x11/0x20
> > .slowpath+0x9/0xe
> > pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:559 [inline]
> > queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
> > lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
> > debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
> > print_usage_bug kernel/locking/lockdep.c:3710 [inline]
>
> Ha, I think you hit a bug in lockdep. But it was about to tell you you
> can't go take locks from NMI context that are also used outside of it.
Mkay. Perf calls a BPF program from NMI context. Should that program
type be significantly restricted? But even if maps can't be used, is
there anything useful a program invoked from such context can do at
all?
> > verify_lock_unused kernel/locking/lockdep.c:5374 [inline]
> > lock_acquire kernel/locking/lockdep.c:5433 [inline]
> > lock_acquire+0x471/0x720 kernel/locking/lockdep.c:5407
> > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline]
> > _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159
> > htab_lock_bucket kernel/bpf/hashtab.c:175 [inline]
> > htab_map_update_elem+0x1f0/0x790 kernel/bpf/hashtab.c:1023
> > bpf_prog_60236c52b8017ad1+0x8e/0xab4
> > bpf_dispatcher_nop_func include/linux/bpf.h:651 [inline]
> > bpf_overflow_handler+0x192/0x5b0 kernel/events/core.c:9755
> > __perf_event_overflow+0x13c/0x370 kernel/events/core.c:8979
> > perf_swevent_overflow kernel/events/core.c:9055 [inline]
> > perf_swevent_event+0x347/0x550 kernel/events/core.c:9083
> > perf_bp_event+0x1a2/0x1c0 kernel/events/core.c:9932
> > hw_breakpoint_handler arch/x86/kernel/hw_breakpoint.c:535 [inline]
> > hw_breakpoint_exceptions_notify+0x18a/0x3b0 arch/x86/kernel/hw_breakpoint.c:567
> > notifier_call_chain+0xb5/0x200 kernel/notifier.c:83
> > atomic_notifier_call_chain+0x8d/0x170 kernel/notifier.c:217
> > notify_die+0xda/0x170 kernel/notifier.c:548
> > notify_debug+0x20/0x30 arch/x86/kernel/traps.c:842
> > exc_debug_kernel arch/x86/kernel/traps.c:902 [inline]
> > exc_debug+0x103/0x140 arch/x86/kernel/traps.c:998
> > asm_exc_debug+0x19/0x30 arch/x86/include/asm/idtentry.h:598
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem
2021-02-01 9:50 ` Peter Zijlstra
2021-02-01 10:06 ` Dmitry Vyukov
@ 2021-02-01 11:23 ` Peter Zijlstra
2021-02-01 17:53 ` Waiman Long
2021-02-08 12:06 ` [tip: locking/core] locking/lockdep: Avoid unmatched unlock tip-bot2 for Peter Zijlstra
1 sibling, 2 replies; 8+ messages in thread
From: Peter Zijlstra @ 2021-02-01 11:23 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau,
Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf,
LKML, Ingo Molnar
On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote:
> > queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
> > lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
> > debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
> > print_usage_bug kernel/locking/lockdep.c:3710 [inline]
>
> Ha, I think you hit a bug in lockdep.
Something like so I suppose.
---
Subject: locking/lockdep: Avoid unmatched unlock
From: Peter Zijlstra <peterz@infradead.org>
Date: Mon Feb 1 11:55:38 CET 2021
Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI"
inversions") overlooked that print_usage_bug() releases the graph_lock
and called it without the graph lock held.
Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions")
Reported-by: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
---
kernel/locking/lockdep.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -3773,7 +3773,7 @@ static void
print_usage_bug(struct task_struct *curr, struct held_lock *this,
enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit)
{
- if (!debug_locks_off_graph_unlock() || debug_locks_silent)
+ if (!debug_locks_off() || debug_locks_silent)
return;
pr_warn("\n");
@@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st
enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit)
{
if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) {
+ graph_unlock()
print_usage_bug(curr, this, bad_bit, new_bit);
return 0;
}
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem
2021-02-01 11:23 ` Peter Zijlstra
@ 2021-02-01 17:53 ` Waiman Long
2021-02-01 18:09 ` Dmitry Vyukov
2021-02-08 12:06 ` [tip: locking/core] locking/lockdep: Avoid unmatched unlock tip-bot2 for Peter Zijlstra
1 sibling, 1 reply; 8+ messages in thread
From: Waiman Long @ 2021-02-01 17:53 UTC (permalink / raw)
To: Peter Zijlstra, Dmitry Vyukov
Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau,
Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf,
LKML, Ingo Molnar
On 2/1/21 6:23 AM, Peter Zijlstra wrote:
> On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote:
>
>>> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
>>> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
>>> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
>>> print_usage_bug kernel/locking/lockdep.c:3710 [inline]
>> Ha, I think you hit a bug in lockdep.
> Something like so I suppose.
>
> ---
> Subject: locking/lockdep: Avoid unmatched unlock
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Mon Feb 1 11:55:38 CET 2021
>
> Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI"
> inversions") overlooked that print_usage_bug() releases the graph_lock
> and called it without the graph lock held.
>
> Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions")
> Reported-by: Dmitry Vyukov <dvyukov@google.com>
> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> ---
> kernel/locking/lockdep.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> --- a/kernel/locking/lockdep.c
> +++ b/kernel/locking/lockdep.c
> @@ -3773,7 +3773,7 @@ static void
> print_usage_bug(struct task_struct *curr, struct held_lock *this,
> enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit)
> {
> - if (!debug_locks_off_graph_unlock() || debug_locks_silent)
> + if (!debug_locks_off() || debug_locks_silent)
> return;
>
> pr_warn("\n");
> @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st
> enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit)
> {
> if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) {
> + graph_unlock()
> print_usage_bug(curr, this, bad_bit, new_bit);
> return 0;
> }
I have also suspected doing unlock without a corresponding lock. This
patch looks good to me.
Acked-by: Waiman Long <longman@redhat.com>
Cheers,
Longman
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem
2021-02-01 17:53 ` Waiman Long
@ 2021-02-01 18:09 ` Dmitry Vyukov
2021-02-01 18:14 ` Waiman Long
0 siblings, 1 reply; 8+ messages in thread
From: Dmitry Vyukov @ 2021-02-01 18:09 UTC (permalink / raw)
To: Waiman Long
Cc: Peter Zijlstra, Alexei Starovoitov, Daniel Borkmann, andrii,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
kpsingh, netdev, bpf, LKML, Ingo Molnar
On Mon, Feb 1, 2021 at 6:54 PM Waiman Long <longman@redhat.com> wrote:
>
> On 2/1/21 6:23 AM, Peter Zijlstra wrote:
> > On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote:
> >
> >>> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
> >>> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
> >>> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
> >>> print_usage_bug kernel/locking/lockdep.c:3710 [inline]
> >> Ha, I think you hit a bug in lockdep.
> > Something like so I suppose.
> >
> > ---
> > Subject: locking/lockdep: Avoid unmatched unlock
> > From: Peter Zijlstra <peterz@infradead.org>
> > Date: Mon Feb 1 11:55:38 CET 2021
> >
> > Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI"
> > inversions") overlooked that print_usage_bug() releases the graph_lock
> > and called it without the graph lock held.
> >
> > Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions")
> > Reported-by: Dmitry Vyukov <dvyukov@google.com>
> > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
> > ---
> > kernel/locking/lockdep.c | 3 ++-
> > 1 file changed, 2 insertions(+), 1 deletion(-)
> >
> > --- a/kernel/locking/lockdep.c
> > +++ b/kernel/locking/lockdep.c
> > @@ -3773,7 +3773,7 @@ static void
> > print_usage_bug(struct task_struct *curr, struct held_lock *this,
> > enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit)
> > {
> > - if (!debug_locks_off_graph_unlock() || debug_locks_silent)
> > + if (!debug_locks_off() || debug_locks_silent)
> > return;
> >
> > pr_warn("\n");
> > @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st
> > enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit)
> > {
> > if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) {
> > + graph_unlock()
> > print_usage_bug(curr, this, bad_bit, new_bit);
> > return 0;
> > }
>
> I have also suspected doing unlock without a corresponding lock. This
> patch looks good to me.
>
> Acked-by: Waiman Long <longman@redhat.com>
Just so that it's not lost: there is still a bug related to bpf map lock, right?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem
2021-02-01 18:09 ` Dmitry Vyukov
@ 2021-02-01 18:14 ` Waiman Long
0 siblings, 0 replies; 8+ messages in thread
From: Waiman Long @ 2021-02-01 18:14 UTC (permalink / raw)
To: Dmitry Vyukov
Cc: Peter Zijlstra, Alexei Starovoitov, Daniel Borkmann, andrii,
Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend,
kpsingh, netdev, bpf, LKML, Ingo Molnar
On 2/1/21 1:09 PM, Dmitry Vyukov wrote:
> On Mon, Feb 1, 2021 at 6:54 PM Waiman Long <longman@redhat.com> wrote:
>> On 2/1/21 6:23 AM, Peter Zijlstra wrote:
>>> On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote:
>>>
>>>>> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline]
>>>>> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124
>>>>> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline]
>>>>> print_usage_bug kernel/locking/lockdep.c:3710 [inline]
>>>> Ha, I think you hit a bug in lockdep.
>>> Something like so I suppose.
>>>
>>> ---
>>> Subject: locking/lockdep: Avoid unmatched unlock
>>> From: Peter Zijlstra <peterz@infradead.org>
>>> Date: Mon Feb 1 11:55:38 CET 2021
>>>
>>> Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI"
>>> inversions") overlooked that print_usage_bug() releases the graph_lock
>>> and called it without the graph lock held.
>>>
>>> Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions")
>>> Reported-by: Dmitry Vyukov <dvyukov@google.com>
>>> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
>>> ---
>>> kernel/locking/lockdep.c | 3 ++-
>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>
>>> --- a/kernel/locking/lockdep.c
>>> +++ b/kernel/locking/lockdep.c
>>> @@ -3773,7 +3773,7 @@ static void
>>> print_usage_bug(struct task_struct *curr, struct held_lock *this,
>>> enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit)
>>> {
>>> - if (!debug_locks_off_graph_unlock() || debug_locks_silent)
>>> + if (!debug_locks_off() || debug_locks_silent)
>>> return;
>>>
>>> pr_warn("\n");
>>> @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st
>>> enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit)
>>> {
>>> if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) {
>>> + graph_unlock()
>>> print_usage_bug(curr, this, bad_bit, new_bit);
>>> return 0;
>>> }
>> I have also suspected doing unlock without a corresponding lock. This
>> patch looks good to me.
>>
>> Acked-by: Waiman Long <longman@redhat.com>
> Just so that it's not lost: there is still a bug related to bpf map lock, right?
>
That is right. This patch just fixes the bug in lockdep.
Cheers,
Longman
^ permalink raw reply [flat|nested] 8+ messages in thread
* [tip: locking/core] locking/lockdep: Avoid unmatched unlock
2021-02-01 11:23 ` Peter Zijlstra
2021-02-01 17:53 ` Waiman Long
@ 2021-02-08 12:06 ` tip-bot2 for Peter Zijlstra
1 sibling, 0 replies; 8+ messages in thread
From: tip-bot2 for Peter Zijlstra @ 2021-02-08 12:06 UTC (permalink / raw)
To: linux-tip-commits
Cc: Dmitry Vyukov, Peter Zijlstra (Intel), Waiman Long, x86, linux-kernel
The following commit has been merged into the locking/core branch of tip:
Commit-ID: 7f82e631d236cafd28518b998c6d4d8dc2ef68f6
Gitweb: https://git.kernel.org/tip/7f82e631d236cafd28518b998c6d4d8dc2ef68f6
Author: Peter Zijlstra <peterz@infradead.org>
AuthorDate: Mon, 01 Feb 2021 11:55:38 +01:00
Committer: Peter Zijlstra <peterz@infradead.org>
CommitterDate: Fri, 05 Feb 2021 17:20:15 +01:00
locking/lockdep: Avoid unmatched unlock
Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI"
inversions") overlooked that print_usage_bug() releases the graph_lock
and called it without the graph lock held.
Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions")
Reported-by: Dmitry Vyukov <dvyukov@google.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Acked-by: Waiman Long <longman@redhat.com>
Link: https://lkml.kernel.org/r/YBfkuyIfB1+VRxXP@hirez.programming.kicks-ass.net
---
kernel/locking/lockdep.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/kernel/locking/lockdep.c b/kernel/locking/lockdep.c
index ad9afd8..5104db0 100644
--- a/kernel/locking/lockdep.c
+++ b/kernel/locking/lockdep.c
@@ -3773,7 +3773,7 @@ static void
print_usage_bug(struct task_struct *curr, struct held_lock *this,
enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit)
{
- if (!debug_locks_off_graph_unlock() || debug_locks_silent)
+ if (!debug_locks_off() || debug_locks_silent)
return;
pr_warn("\n");
@@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, struct held_lock *this,
enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit)
{
if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) {
+ graph_unlock();
print_usage_bug(curr, this, bad_bit, new_bit);
return 0;
}
^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-02-08 12:24 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-01-31 8:42 corrupted pvqspinlock in htab_map_update_elem Dmitry Vyukov
2021-02-01 9:50 ` Peter Zijlstra
2021-02-01 10:06 ` Dmitry Vyukov
2021-02-01 11:23 ` Peter Zijlstra
2021-02-01 17:53 ` Waiman Long
2021-02-01 18:09 ` Dmitry Vyukov
2021-02-01 18:14 ` Waiman Long
2021-02-08 12:06 ` [tip: locking/core] locking/lockdep: Avoid unmatched unlock tip-bot2 for Peter Zijlstra
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.