* corrupted pvqspinlock in htab_map_update_elem @ 2021-01-31 8:42 Dmitry Vyukov 2021-02-01 9:50 ` Peter Zijlstra 0 siblings, 1 reply; 7+ messages in thread From: Dmitry Vyukov @ 2021-01-31 8:42 UTC (permalink / raw) To: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Peter Zijlstra, Ingo Molnar Hi, I am testing the following the program: https://gist.github.com/dvyukov/e5c0a8ef220ef856363c1080b0936a9e on the latest upstream 6642d600b541b81931fb1ab0c041b0d68f77be7e and getting the following crash. Config is: https://gist.github.com/dvyukov/16d9905e5ef35e44285451f1d330ddbc The program updates a bpf map from a program called on hw breakpoint hit. Not sure if it's a bpf issue or a perf issue. This time it is not a fuzzer workload, I am trying to do something useful :) ------------[ cut here ]------------ pvqspinlock: lock 0xffffffff8f371d80 has corrupted value 0x0! WARNING: CPU: 3 PID: 8771 at kernel/locking/qspinlock_paravirt.h:498 __pv_queued_spin_unlock_slowpath+0x22e/0x2b0 kernel/locking/qspinlock_paravirt.h:498 Modules linked in: CPU: 3 PID: 8771 Comm: a.out Not tainted 5.11.0-rc5+ #71 Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS rel-1.13.0-44-g88ab0c15525c-prebuilt.qemu.org 04/01/2014 RIP: 0010:__pv_queued_spin_unlock_slowpath+0x22e/0x2b0 kernel/locking/qspinlock_paravirt.h:498 Code: ea 03 0f b6 14 02 4c 89 e8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2 75 62 41 8b 55 00 4c 89 ee 48 c7 c7 20 6b 4c 89 e8 72 d3 5f 07 <0f> 0b e9 6cc RSP: 0018:fffffe00000c17b0 EFLAGS: 00010086 RAX: 0000000000000000 RBX: ffffffff8f3b5660 RCX: 0000000000000000 RDX: ffff8880150222c0 RSI: ffffffff815b624d RDI: fffffbc0000182e8 RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000 R10: ffffffff817de94f R11: 0000000000000000 R12: ffff8880150222c0 R13: ffffffff8f371d80 R14: ffff8880181fead8 R15: 0000000000000000 FS: 00007fa5b51f0700(0000) GS:ffff88802cf80000(0000) knlGS:0000000000000000 CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 CR2: 0000000002286908 CR3: 0000000015b24000 CR4: 0000000000750ee0 DR0: 00000000004cb3d4 DR1: 0000000000000000 DR2: 0000000000000000 DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 PKRU: 55555554 Call Trace: <#DB> __raw_callee_save___pv_queued_spin_unlock_slowpath+0x11/0x20 .slowpath+0x9/0xe pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:559 [inline] queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] print_usage_bug kernel/locking/lockdep.c:3710 [inline] verify_lock_unused kernel/locking/lockdep.c:5374 [inline] lock_acquire kernel/locking/lockdep.c:5433 [inline] lock_acquire+0x471/0x720 kernel/locking/lockdep.c:5407 __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159 htab_lock_bucket kernel/bpf/hashtab.c:175 [inline] htab_map_update_elem+0x1f0/0x790 kernel/bpf/hashtab.c:1023 bpf_prog_60236c52b8017ad1+0x8e/0xab4 bpf_dispatcher_nop_func include/linux/bpf.h:651 [inline] bpf_overflow_handler+0x192/0x5b0 kernel/events/core.c:9755 __perf_event_overflow+0x13c/0x370 kernel/events/core.c:8979 perf_swevent_overflow kernel/events/core.c:9055 [inline] perf_swevent_event+0x347/0x550 kernel/events/core.c:9083 perf_bp_event+0x1a2/0x1c0 kernel/events/core.c:9932 hw_breakpoint_handler arch/x86/kernel/hw_breakpoint.c:535 [inline] hw_breakpoint_exceptions_notify+0x18a/0x3b0 arch/x86/kernel/hw_breakpoint.c:567 notifier_call_chain+0xb5/0x200 kernel/notifier.c:83 atomic_notifier_call_chain+0x8d/0x170 kernel/notifier.c:217 notify_die+0xda/0x170 kernel/notifier.c:548 notify_debug+0x20/0x30 arch/x86/kernel/traps.c:842 exc_debug_kernel arch/x86/kernel/traps.c:902 [inline] exc_debug+0x103/0x140 arch/x86/kernel/traps.c:998 asm_exc_debug+0x19/0x30 arch/x86/include/asm/idtentry.h:598 RIP: 0010:copy_user_generic_unrolled+0xa2/0xc0 arch/x86/lib/copy_user_64.S:102 Code: ff c9 75 b6 89 d1 83 e2 07 c1 e9 03 74 12 4c 8b 06 4c 89 07 48 8d 76 08 48 8d 7f 08 ff c9 75 ee 21 d2 74 10 89 d1 8a 06 88 07 <48> ff c6 484 RSP: 0018:ffffc90000d67af0 EFLAGS: 00040202 RAX: 0000000000000001 RBX: 0000000000000001 RCX: 0000000000000001 RDX: 0000000000000001 RSI: ffff88801341d001 RDI: 00000000004cb3d4 RBP: 00000000004cb3d4 R08: 0000000000000000 R09: ffff88801341d001 R10: ffffed1002683a00 R11: 0000000000000000 R12: ffff88801341d001 R13: 00000000004cb3d5 R14: 0000000000000000 R15: 0000000000000001 </#DB> copy_user_generic arch/x86/include/asm/uaccess_64.h:37 [inline] raw_copy_to_user arch/x86/include/asm/uaccess_64.h:58 [inline] copyout.part.0+0xe4/0x110 lib/iov_iter.c:148 copyout lib/iov_iter.c:182 [inline] copy_page_to_iter_iovec lib/iov_iter.c:219 [inline] copy_page_to_iter+0x416/0xed0 lib/iov_iter.c:926 pipe_read+0x4a4/0x13a0 fs/pipe.c:290 call_read_iter include/linux/fs.h:1895 [inline] new_sync_read+0x5b7/0x6e0 fs/read_write.c:415 vfs_read+0x35c/0x570 fs/read_write.c:496 ksys_read+0x1ee/0x250 fs/read_write.c:634 do_syscall_64+0x2d/0x70 arch/x86/entry/common.c:46 entry_SYSCALL_64_after_hwframe+0x44/0xa9 RIP: 0033:0x406c1c Code: ec 28 48 89 54 24 18 48 89 74 24 10 89 7c 24 08 e8 f9 fc ff ff 48 8b 54 24 18 48 8b 74 24 10 41 89 c0 8b 7c 24 08 31 c0 0f 05 <48> 3d 00 f08 RSP: 002b:00007fa5b51f02d0 EFLAGS: 00000246 ORIG_RAX: 0000000000000000 RAX: ffffffffffffffda RBX: 00000000004cb3d4 RCX: 0000000000406c1c RDX: 0000000000000001 RSI: 00000000004cb3d4 RDI: 0000000000000004 RBP: 00007fa5b51f030c R08: 0000000000000000 R09: 0000000000000000 R10: 00007fa5b51f0700 R11: 0000000000000246 R12: 00000000004cb3d0 R13: cccccccccccccccd R14: 00007fa5b51f0400 R15: 0000000000802000 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem 2021-01-31 8:42 corrupted pvqspinlock in htab_map_update_elem Dmitry Vyukov @ 2021-02-01 9:50 ` Peter Zijlstra 2021-02-01 10:06 ` Dmitry Vyukov 2021-02-01 11:23 ` Peter Zijlstra 0 siblings, 2 replies; 7+ messages in thread From: Peter Zijlstra @ 2021-02-01 9:50 UTC (permalink / raw) To: Dmitry Vyukov Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Ingo Molnar On Sun, Jan 31, 2021 at 09:42:53AM +0100, Dmitry Vyukov wrote: > Hi, > > I am testing the following the program: > https://gist.github.com/dvyukov/e5c0a8ef220ef856363c1080b0936a9e > on the latest upstream 6642d600b541b81931fb1ab0c041b0d68f77be7e and > getting the following crash. Config is: > https://gist.github.com/dvyukov/16d9905e5ef35e44285451f1d330ddbc > > The program updates a bpf map from a program called on hw breakpoint > hit. Not sure if it's a bpf issue or a perf issue. This time it is not > a fuzzer workload, I am trying to do something useful :) Something useful and BPF don't go together as far as I'm concerned. > ------------[ cut here ]------------ > pvqspinlock: lock 0xffffffff8f371d80 has corrupted value 0x0! > WARNING: CPU: 3 PID: 8771 at kernel/locking/qspinlock_paravirt.h:498 > __pv_queued_spin_unlock_slowpath+0x22e/0x2b0 > kernel/locking/qspinlock_paravirt.h:498 > Modules linked in: > CPU: 3 PID: 8771 Comm: a.out Not tainted 5.11.0-rc5+ #71 > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS > rel-1.13.0-44-g88ab0c15525c-prebuilt.qemu.org 04/01/2014 > RIP: 0010:__pv_queued_spin_unlock_slowpath+0x22e/0x2b0 > kernel/locking/qspinlock_paravirt.h:498 > Code: ea 03 0f b6 14 02 4c 89 e8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2 > 75 62 41 8b 55 00 4c 89 ee 48 c7 c7 20 6b 4c 89 e8 72 d3 5f 07 <0f> 0b > e9 6cc > RSP: 0018:fffffe00000c17b0 EFLAGS: 00010086 > RAX: 0000000000000000 RBX: ffffffff8f3b5660 RCX: 0000000000000000 > RDX: ffff8880150222c0 RSI: ffffffff815b624d RDI: fffffbc0000182e8 > RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000 > R10: ffffffff817de94f R11: 0000000000000000 R12: ffff8880150222c0 > R13: ffffffff8f371d80 R14: ffff8880181fead8 R15: 0000000000000000 > FS: 00007fa5b51f0700(0000) GS:ffff88802cf80000(0000) knlGS:0000000000000000 > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > CR2: 0000000002286908 CR3: 0000000015b24000 CR4: 0000000000750ee0 > DR0: 00000000004cb3d4 DR1: 0000000000000000 DR2: 0000000000000000 > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > PKRU: 55555554 > Call Trace: > <#DB> > __raw_callee_save___pv_queued_spin_unlock_slowpath+0x11/0x20 > .slowpath+0x9/0xe > pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:559 [inline] > queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] > lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 > debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] > print_usage_bug kernel/locking/lockdep.c:3710 [inline] Ha, I think you hit a bug in lockdep. But it was about to tell you you can't go take locks from NMI context that are also used outside of it. > verify_lock_unused kernel/locking/lockdep.c:5374 [inline] > lock_acquire kernel/locking/lockdep.c:5433 [inline] > lock_acquire+0x471/0x720 kernel/locking/lockdep.c:5407 > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] > _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159 > htab_lock_bucket kernel/bpf/hashtab.c:175 [inline] > htab_map_update_elem+0x1f0/0x790 kernel/bpf/hashtab.c:1023 > bpf_prog_60236c52b8017ad1+0x8e/0xab4 > bpf_dispatcher_nop_func include/linux/bpf.h:651 [inline] > bpf_overflow_handler+0x192/0x5b0 kernel/events/core.c:9755 > __perf_event_overflow+0x13c/0x370 kernel/events/core.c:8979 > perf_swevent_overflow kernel/events/core.c:9055 [inline] > perf_swevent_event+0x347/0x550 kernel/events/core.c:9083 > perf_bp_event+0x1a2/0x1c0 kernel/events/core.c:9932 > hw_breakpoint_handler arch/x86/kernel/hw_breakpoint.c:535 [inline] > hw_breakpoint_exceptions_notify+0x18a/0x3b0 arch/x86/kernel/hw_breakpoint.c:567 > notifier_call_chain+0xb5/0x200 kernel/notifier.c:83 > atomic_notifier_call_chain+0x8d/0x170 kernel/notifier.c:217 > notify_die+0xda/0x170 kernel/notifier.c:548 > notify_debug+0x20/0x30 arch/x86/kernel/traps.c:842 > exc_debug_kernel arch/x86/kernel/traps.c:902 [inline] > exc_debug+0x103/0x140 arch/x86/kernel/traps.c:998 > asm_exc_debug+0x19/0x30 arch/x86/include/asm/idtentry.h:598 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem 2021-02-01 9:50 ` Peter Zijlstra @ 2021-02-01 10:06 ` Dmitry Vyukov 2021-02-01 11:23 ` Peter Zijlstra 1 sibling, 0 replies; 7+ messages in thread From: Dmitry Vyukov @ 2021-02-01 10:06 UTC (permalink / raw) To: Peter Zijlstra Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Ingo Molnar On Mon, Feb 1, 2021 at 10:51 AM Peter Zijlstra <peterz@infradead.org> wrote: > > On Sun, Jan 31, 2021 at 09:42:53AM +0100, Dmitry Vyukov wrote: > > Hi, > > > > I am testing the following the program: > > https://gist.github.com/dvyukov/e5c0a8ef220ef856363c1080b0936a9e > > on the latest upstream 6642d600b541b81931fb1ab0c041b0d68f77be7e and > > getting the following crash. Config is: > > https://gist.github.com/dvyukov/16d9905e5ef35e44285451f1d330ddbc > > > > The program updates a bpf map from a program called on hw breakpoint > > hit. Not sure if it's a bpf issue or a perf issue. This time it is not > > a fuzzer workload, I am trying to do something useful :) > > Something useful and BPF don't go together as far as I'm concerned. > > > ------------[ cut here ]------------ > > pvqspinlock: lock 0xffffffff8f371d80 has corrupted value 0x0! > > WARNING: CPU: 3 PID: 8771 at kernel/locking/qspinlock_paravirt.h:498 > > __pv_queued_spin_unlock_slowpath+0x22e/0x2b0 > > kernel/locking/qspinlock_paravirt.h:498 > > Modules linked in: > > CPU: 3 PID: 8771 Comm: a.out Not tainted 5.11.0-rc5+ #71 > > Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS > > rel-1.13.0-44-g88ab0c15525c-prebuilt.qemu.org 04/01/2014 > > RIP: 0010:__pv_queued_spin_unlock_slowpath+0x22e/0x2b0 > > kernel/locking/qspinlock_paravirt.h:498 > > Code: ea 03 0f b6 14 02 4c 89 e8 83 e0 07 83 c0 03 38 d0 7c 04 84 d2 > > 75 62 41 8b 55 00 4c 89 ee 48 c7 c7 20 6b 4c 89 e8 72 d3 5f 07 <0f> 0b > > e9 6cc > > RSP: 0018:fffffe00000c17b0 EFLAGS: 00010086 > > RAX: 0000000000000000 RBX: ffffffff8f3b5660 RCX: 0000000000000000 > > RDX: ffff8880150222c0 RSI: ffffffff815b624d RDI: fffffbc0000182e8 > > RBP: 0000000000000001 R08: 0000000000000001 R09: 0000000000000000 > > R10: ffffffff817de94f R11: 0000000000000000 R12: ffff8880150222c0 > > R13: ffffffff8f371d80 R14: ffff8880181fead8 R15: 0000000000000000 > > FS: 00007fa5b51f0700(0000) GS:ffff88802cf80000(0000) knlGS:0000000000000000 > > CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > > CR2: 0000000002286908 CR3: 0000000015b24000 CR4: 0000000000750ee0 > > DR0: 00000000004cb3d4 DR1: 0000000000000000 DR2: 0000000000000000 > > DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 > > PKRU: 55555554 > > Call Trace: > > <#DB> > > __raw_callee_save___pv_queued_spin_unlock_slowpath+0x11/0x20 > > .slowpath+0x9/0xe > > pv_queued_spin_unlock arch/x86/include/asm/paravirt.h:559 [inline] > > queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] > > lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 > > debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] > > print_usage_bug kernel/locking/lockdep.c:3710 [inline] > > Ha, I think you hit a bug in lockdep. But it was about to tell you you > can't go take locks from NMI context that are also used outside of it. Mkay. Perf calls a BPF program from NMI context. Should that program type be significantly restricted? But even if maps can't be used, is there anything useful a program invoked from such context can do at all? > > verify_lock_unused kernel/locking/lockdep.c:5374 [inline] > > lock_acquire kernel/locking/lockdep.c:5433 [inline] > > lock_acquire+0x471/0x720 kernel/locking/lockdep.c:5407 > > __raw_spin_lock_irqsave include/linux/spinlock_api_smp.h:110 [inline] > > _raw_spin_lock_irqsave+0x39/0x50 kernel/locking/spinlock.c:159 > > htab_lock_bucket kernel/bpf/hashtab.c:175 [inline] > > htab_map_update_elem+0x1f0/0x790 kernel/bpf/hashtab.c:1023 > > bpf_prog_60236c52b8017ad1+0x8e/0xab4 > > bpf_dispatcher_nop_func include/linux/bpf.h:651 [inline] > > bpf_overflow_handler+0x192/0x5b0 kernel/events/core.c:9755 > > __perf_event_overflow+0x13c/0x370 kernel/events/core.c:8979 > > perf_swevent_overflow kernel/events/core.c:9055 [inline] > > perf_swevent_event+0x347/0x550 kernel/events/core.c:9083 > > perf_bp_event+0x1a2/0x1c0 kernel/events/core.c:9932 > > hw_breakpoint_handler arch/x86/kernel/hw_breakpoint.c:535 [inline] > > hw_breakpoint_exceptions_notify+0x18a/0x3b0 arch/x86/kernel/hw_breakpoint.c:567 > > notifier_call_chain+0xb5/0x200 kernel/notifier.c:83 > > atomic_notifier_call_chain+0x8d/0x170 kernel/notifier.c:217 > > notify_die+0xda/0x170 kernel/notifier.c:548 > > notify_debug+0x20/0x30 arch/x86/kernel/traps.c:842 > > exc_debug_kernel arch/x86/kernel/traps.c:902 [inline] > > exc_debug+0x103/0x140 arch/x86/kernel/traps.c:998 > > asm_exc_debug+0x19/0x30 arch/x86/include/asm/idtentry.h:598 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem 2021-02-01 9:50 ` Peter Zijlstra 2021-02-01 10:06 ` Dmitry Vyukov @ 2021-02-01 11:23 ` Peter Zijlstra 2021-02-01 17:53 ` Waiman Long 1 sibling, 1 reply; 7+ messages in thread From: Peter Zijlstra @ 2021-02-01 11:23 UTC (permalink / raw) To: Dmitry Vyukov Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Ingo Molnar On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote: > > queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] > > lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 > > debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] > > print_usage_bug kernel/locking/lockdep.c:3710 [inline] > > Ha, I think you hit a bug in lockdep. Something like so I suppose. --- Subject: locking/lockdep: Avoid unmatched unlock From: Peter Zijlstra <peterz@infradead.org> Date: Mon Feb 1 11:55:38 CET 2021 Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions") overlooked that print_usage_bug() releases the graph_lock and called it without the graph lock held. Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions") Reported-by: Dmitry Vyukov <dvyukov@google.com> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> --- kernel/locking/lockdep.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/kernel/locking/lockdep.c +++ b/kernel/locking/lockdep.c @@ -3773,7 +3773,7 @@ static void print_usage_bug(struct task_struct *curr, struct held_lock *this, enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit) { - if (!debug_locks_off_graph_unlock() || debug_locks_silent) + if (!debug_locks_off() || debug_locks_silent) return; pr_warn("\n"); @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit) { if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) { + graph_unlock() print_usage_bug(curr, this, bad_bit, new_bit); return 0; } ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem 2021-02-01 11:23 ` Peter Zijlstra @ 2021-02-01 17:53 ` Waiman Long 2021-02-01 18:09 ` Dmitry Vyukov 0 siblings, 1 reply; 7+ messages in thread From: Waiman Long @ 2021-02-01 17:53 UTC (permalink / raw) To: Peter Zijlstra, Dmitry Vyukov Cc: Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Ingo Molnar On 2/1/21 6:23 AM, Peter Zijlstra wrote: > On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote: > >>> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] >>> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 >>> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] >>> print_usage_bug kernel/locking/lockdep.c:3710 [inline] >> Ha, I think you hit a bug in lockdep. > Something like so I suppose. > > --- > Subject: locking/lockdep: Avoid unmatched unlock > From: Peter Zijlstra <peterz@infradead.org> > Date: Mon Feb 1 11:55:38 CET 2021 > > Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" > inversions") overlooked that print_usage_bug() releases the graph_lock > and called it without the graph lock held. > > Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions") > Reported-by: Dmitry Vyukov <dvyukov@google.com> > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> > --- > kernel/locking/lockdep.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > --- a/kernel/locking/lockdep.c > +++ b/kernel/locking/lockdep.c > @@ -3773,7 +3773,7 @@ static void > print_usage_bug(struct task_struct *curr, struct held_lock *this, > enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit) > { > - if (!debug_locks_off_graph_unlock() || debug_locks_silent) > + if (!debug_locks_off() || debug_locks_silent) > return; > > pr_warn("\n"); > @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st > enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit) > { > if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) { > + graph_unlock() > print_usage_bug(curr, this, bad_bit, new_bit); > return 0; > } I have also suspected doing unlock without a corresponding lock. This patch looks good to me. Acked-by: Waiman Long <longman@redhat.com> Cheers, Longman ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem 2021-02-01 17:53 ` Waiman Long @ 2021-02-01 18:09 ` Dmitry Vyukov 2021-02-01 18:14 ` Waiman Long 0 siblings, 1 reply; 7+ messages in thread From: Dmitry Vyukov @ 2021-02-01 18:09 UTC (permalink / raw) To: Waiman Long Cc: Peter Zijlstra, Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Ingo Molnar On Mon, Feb 1, 2021 at 6:54 PM Waiman Long <longman@redhat.com> wrote: > > On 2/1/21 6:23 AM, Peter Zijlstra wrote: > > On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote: > > > >>> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] > >>> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 > >>> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] > >>> print_usage_bug kernel/locking/lockdep.c:3710 [inline] > >> Ha, I think you hit a bug in lockdep. > > Something like so I suppose. > > > > --- > > Subject: locking/lockdep: Avoid unmatched unlock > > From: Peter Zijlstra <peterz@infradead.org> > > Date: Mon Feb 1 11:55:38 CET 2021 > > > > Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" > > inversions") overlooked that print_usage_bug() releases the graph_lock > > and called it without the graph lock held. > > > > Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions") > > Reported-by: Dmitry Vyukov <dvyukov@google.com> > > Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> > > --- > > kernel/locking/lockdep.c | 3 ++- > > 1 file changed, 2 insertions(+), 1 deletion(-) > > > > --- a/kernel/locking/lockdep.c > > +++ b/kernel/locking/lockdep.c > > @@ -3773,7 +3773,7 @@ static void > > print_usage_bug(struct task_struct *curr, struct held_lock *this, > > enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit) > > { > > - if (!debug_locks_off_graph_unlock() || debug_locks_silent) > > + if (!debug_locks_off() || debug_locks_silent) > > return; > > > > pr_warn("\n"); > > @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st > > enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit) > > { > > if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) { > > + graph_unlock() > > print_usage_bug(curr, this, bad_bit, new_bit); > > return 0; > > } > > I have also suspected doing unlock without a corresponding lock. This > patch looks good to me. > > Acked-by: Waiman Long <longman@redhat.com> Just so that it's not lost: there is still a bug related to bpf map lock, right? ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: corrupted pvqspinlock in htab_map_update_elem 2021-02-01 18:09 ` Dmitry Vyukov @ 2021-02-01 18:14 ` Waiman Long 0 siblings, 0 replies; 7+ messages in thread From: Waiman Long @ 2021-02-01 18:14 UTC (permalink / raw) To: Dmitry Vyukov Cc: Peter Zijlstra, Alexei Starovoitov, Daniel Borkmann, andrii, Martin KaFai Lau, Song Liu, Yonghong Song, John Fastabend, kpsingh, netdev, bpf, LKML, Ingo Molnar On 2/1/21 1:09 PM, Dmitry Vyukov wrote: > On Mon, Feb 1, 2021 at 6:54 PM Waiman Long <longman@redhat.com> wrote: >> On 2/1/21 6:23 AM, Peter Zijlstra wrote: >>> On Mon, Feb 01, 2021 at 10:50:58AM +0100, Peter Zijlstra wrote: >>> >>>>> queued_spin_unlock arch/x86/include/asm/qspinlock.h:56 [inline] >>>>> lockdep_unlock+0x10e/0x290 kernel/locking/lockdep.c:124 >>>>> debug_locks_off_graph_unlock kernel/locking/lockdep.c:165 [inline] >>>>> print_usage_bug kernel/locking/lockdep.c:3710 [inline] >>>> Ha, I think you hit a bug in lockdep. >>> Something like so I suppose. >>> >>> --- >>> Subject: locking/lockdep: Avoid unmatched unlock >>> From: Peter Zijlstra <peterz@infradead.org> >>> Date: Mon Feb 1 11:55:38 CET 2021 >>> >>> Commit f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" >>> inversions") overlooked that print_usage_bug() releases the graph_lock >>> and called it without the graph lock held. >>> >>> Fixes: f6f48e180404 ("lockdep: Teach lockdep about "USED" <- "IN-NMI" inversions") >>> Reported-by: Dmitry Vyukov <dvyukov@google.com> >>> Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org> >>> --- >>> kernel/locking/lockdep.c | 3 ++- >>> 1 file changed, 2 insertions(+), 1 deletion(-) >>> >>> --- a/kernel/locking/lockdep.c >>> +++ b/kernel/locking/lockdep.c >>> @@ -3773,7 +3773,7 @@ static void >>> print_usage_bug(struct task_struct *curr, struct held_lock *this, >>> enum lock_usage_bit prev_bit, enum lock_usage_bit new_bit) >>> { >>> - if (!debug_locks_off_graph_unlock() || debug_locks_silent) >>> + if (!debug_locks_off() || debug_locks_silent) >>> return; >>> >>> pr_warn("\n"); >>> @@ -3814,6 +3814,7 @@ valid_state(struct task_struct *curr, st >>> enum lock_usage_bit new_bit, enum lock_usage_bit bad_bit) >>> { >>> if (unlikely(hlock_class(this)->usage_mask & (1 << bad_bit))) { >>> + graph_unlock() >>> print_usage_bug(curr, this, bad_bit, new_bit); >>> return 0; >>> } >> I have also suspected doing unlock without a corresponding lock. This >> patch looks good to me. >> >> Acked-by: Waiman Long <longman@redhat.com> > Just so that it's not lost: there is still a bug related to bpf map lock, right? > That is right. This patch just fixes the bug in lockdep. Cheers, Longman ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-02-01 18:17 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-01-31 8:42 corrupted pvqspinlock in htab_map_update_elem Dmitry Vyukov 2021-02-01 9:50 ` Peter Zijlstra 2021-02-01 10:06 ` Dmitry Vyukov 2021-02-01 11:23 ` Peter Zijlstra 2021-02-01 17:53 ` Waiman Long 2021-02-01 18:09 ` Dmitry Vyukov 2021-02-01 18:14 ` Waiman Long
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).