From: Masami Hiramatsu <mhiramat@kernel.org>
To: Masami Hiramatsu <mhiramat@kernel.org>
Cc: linux-kernel@vger.kernel.org,
Peter Zijlstra <peterz@infradead.org>,
Eddy_Wu@trendmicro.com, x86@kernel.org, davem@davemloft.net,
rostedt@goodmis.org, naveen.n.rao@linux.ibm.com,
anil.s.keshavamurthy@intel.com, linux-arch@vger.kernel.org,
cameron@moodycamel.com, oleg@redhat.com, will@kernel.org,
paulmck@kernel.org
Subject: Re: [PATCH v4 19/23] kprobes: Remove kretprobe hash
Date: Sat, 29 Aug 2020 03:37:26 +0900 [thread overview]
Message-ID: <20200829033726.68547b37624d3510ebc33ab1@kernel.org> (raw)
In-Reply-To: <159861780638.992023.16486601398173945135.stgit@devnote2>
On Fri, 28 Aug 2020 21:30:06 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:
> From: Peter Zijlstra <peterz@infradead.org>
>
> The kretprobe hash is mostly superfluous, replace it with a per-task
> variable.
>
> This gets rid of the task hash and it's related locking.
>
> The whole invalidate_rp_inst() is tedious and could go away once we
> drop rp specific ri size.
OK, something wrong with this patch. Now I can reproduce it always,
it takes around 330 second from setting up the kretprobe event on
schedule(). Maybe some timer or timeout value affect?
Anyway, before this patch, I can not reproduce the bug yet. After
applying this patch, I hit the bug always. So something wrong with this.
----
cd /sys/kernel/debug/tracing/
echo r:schedule schedule >> kprobe_events
echo 1 > events/kprobes/enable
sleep 333
----
And show this.
/sys/kernel/debug/tracing # [ 336.718043] ------------[ cut here ]------------
[ 336.719041] kernel BUG at kernel/kprobes.c:1950!
[ 336.720212] invalid opcode: 0000 [#1] PREEMPT SMP PTI
[ 336.721206] CPU: 0 PID: 85 Comm: kworker/0:2 Not tainted 5.9.0-rc2+ #58
[ 336.722557] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 1.13.0-1ubuntu1 04/01/2014
[ 336.724310] Workqueue: 0x0 (events)
[ 336.726997] RIP: 0010:__kretprobe_trampoline_handler+0xe5/0xf0
[ 336.728084] Code: e8 20 04 79 00 65 48 c7 05 ec 4a ec 7e 00 cd 28 82 4c 89 e7 e8 ac fe ff ff 48 85 db 75 97 5b 4c 89 e8 41 5c 41 5d 41 5e 5d c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 49 89 fc 53 48
[ 336.731574] RSP: 0018:ffffc90000307dc8 EFLAGS: 00010246
[ 336.732608] RAX: ffff88807d600000 RBX: 0000000000000000 RCX: 0000000000000000
[ 336.733896] RDX: ffffc90000307ea8 RSI: ffffffff810471e0 RDI: ffff88807c579700
[ 336.735205] RBP: ffffc90000307de8 R08: 0000000000000001 R09: 0000000000000001
[ 336.736506] R10: 0000000000000000 R11: 0000000000000001 R12: ffffc90000307e10
[ 336.737820] R13: ffff88807d62a440 R14: ffffc90000307e10 R15: ffff88807cfb5300
[ 336.739125] FS: 0000000000000000(0000) GS:ffff88807d600000(0000) knlGS:0000000000000000
[ 336.740705] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 336.741773] CR2: 000000000212cd38 CR3: 000000007a732000 CR4: 00000000000006b0
[ 336.742953] Call Trace:
[ 336.743523] trampoline_handler+0x43/0x60
[ 336.744317] kretprobe_trampoline+0x2a/0x50
[ 336.745098] RIP: 0010:kretprobe_trampoline+0x0/0x50
[ 336.745961] Code: c7 39 2e 04 82 e8 a0 f2 0d 00 5d c3 31 f6 e9 79 ff ff ff be 01 00 00 00 e9 6f ff ff ff cc cc cc cc cc cc cc cc cc cc cc cc cc <54> 9c 48 83 ec 18 57 56 52 51 50 41 50 41 51 41 52 41 53 53 55 41
[ 336.749100] RSP: 7c579700:ffffc90000307eb0 EFLAGS: 00000246
[ 336.750077] RAX: ffff88807cfb5300 RBX: ffff88807d62a440 RCX: 0000000000000000
[ 336.751233] RDX: 0000000000000000 RSI: ffffffff818e5334 RDI: ffff88807c579700
[ 336.752391] RBP: ffffc90000307f00 R08: 0000000000000001 R09: 0000000000000001
[ 336.753563] R10: 0000000000000000 R11: 0000000000000001 R12: ffff88807cfb5328
[ 336.754718] R13: ffff88807d62a440 R14: ffffc90000033d58 R15: ffff88807cfb5300
[ 336.756005] ? schedule+0x54/0x100
[ 336.756684] ? process_one_work+0x5c0/0x5c0
[ 336.757504] kthread+0x13c/0x180
[ 336.758102] ? kthread_park+0x90/0x90
[ 336.758931] ret_from_fork+0x22/0x30
[ 336.759719] Modules linked in:
[ 336.760456] ---[ end trace a7f6025840267136 ]---
[ 336.761448] RIP: 0010:__kretprobe_trampoline_handler+0xe5/0xf0
[ 336.762767] Code: e8 20 04 79 00 65 48 c7 05 ec 4a ec 7e 00 cd 28 82 4c 89 e7 e8 ac fe ff ff 48 85 db 75 97 5b 4c 89 e8 41 5c 41 5d 41 5e 5d c3 <0f> 0b 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 41 54 49 89 fc 53 48
[ 336.766128] RSP: 0018:ffffc90000307dc8 EFLAGS: 00010246
[ 336.767112] RAX: ffff88807d600000 RBX: 0000000000000000 RCX: 0000000000000000
[ 336.768321] RDX: ffffc90000307ea8 RSI: ffffffff810471e0 RDI: ffff88807c579700
[ 336.769539] RBP: ffffc90000307de8 R08: 0000000000000001 R09: 0000000000000001
[ 336.770744] R10: 0000000000000000 R11: 0000000000000001 R12: ffffc90000307e10
[ 336.771947] R13: ffff88807d62a440 R14: ffffc90000307e10 R15: ffff88807cfb5300
[ 336.773201] FS: 0000000000000000(0000) GS:ffff88807d600000(0000) knlGS:0000000000000000
[ 336.774706] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 336.775577] CR2: 000000000212cd38 CR3: 000000007a732000 CR4: 00000000000006b0
[ 336.776699] Kernel panic - not syncing: Fatal exception
[ 336.777829] Kernel Offset: disabled
[ 336.778526] ---[ end Kernel panic - not syncing: Fatal exception ]---
Thank you,
--
Masami Hiramatsu <mhiramat@kernel.org>
next prev parent reply other threads:[~2020-08-28 18:37 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-28 12:26 [PATCH v4 00/23] kprobes: Unify kretprobe trampoline handlers and make kretprobe lockless Masami Hiramatsu
2020-08-28 12:26 ` [PATCH v4 01/23] kprobes: Add generic kretprobe trampoline handler Masami Hiramatsu
2020-08-28 12:27 ` [PATCH v4 02/23] x86/kprobes: Use " Masami Hiramatsu
2020-08-28 12:27 ` [PATCH v4 03/23] arm: kprobes: " Masami Hiramatsu
2020-08-28 12:27 ` [PATCH v4 04/23] arm64: " Masami Hiramatsu
2020-08-28 13:31 ` Mark Rutland
2020-08-28 13:37 ` peterz
2020-08-28 13:48 ` Mark Rutland
2020-08-28 13:58 ` Masami Hiramatsu
2020-08-28 12:27 ` [PATCH v4 05/23] arc: " Masami Hiramatsu
2020-08-28 12:27 ` [PATCH v4 06/23] csky: " Masami Hiramatsu
2020-08-28 12:27 ` [PATCH v4 07/23] ia64: " Masami Hiramatsu
2020-08-28 12:28 ` [PATCH v4 08/23] mips: " Masami Hiramatsu
2020-08-28 12:28 ` [PATCH v4 09/23] parisc: " Masami Hiramatsu
2020-08-28 12:28 ` [PATCH v4 10/23] powerpc: " Masami Hiramatsu
2020-08-28 12:28 ` [PATCH v4 11/23] s390: " Masami Hiramatsu
2020-08-28 12:28 ` [PATCH v4 12/23] sh: " Masami Hiramatsu
2020-08-28 12:29 ` [PATCH v4 13/23] sparc: " Masami Hiramatsu
2020-08-28 12:29 ` [PATCH v4 14/23] kprobes: Remove NMI context check Masami Hiramatsu
2020-08-28 12:29 ` [PATCH v4 15/23] kprobes: Free kretprobe_instance with rcu callback Masami Hiramatsu
2020-08-28 12:29 ` [PATCH v4 16/23] kprobes: Make local used functions static Masami Hiramatsu
2020-08-28 12:29 ` [PATCH v4 17/23] llist: Add nonatomic __llist_add() Masami Hiramatsu
2020-08-28 12:29 ` [PATCH v4 18/23] sched: Fix try_invoke_on_locked_down_task() semantics Masami Hiramatsu
2020-08-29 2:01 ` Masami Hiramatsu
2020-08-29 7:30 ` peterz
2020-08-29 17:31 ` Paul E. McKenney
2020-08-28 12:30 ` [PATCH v4 19/23] kprobes: Remove kretprobe hash Masami Hiramatsu
2020-08-28 18:37 ` Masami Hiramatsu [this message]
2020-08-28 19:02 ` peterz
2020-08-28 12:30 ` [PATCH v4 20/23] [RFC] kprobes: Remove task scan for updating kretprobe_instance Masami Hiramatsu
2020-08-28 12:52 ` peterz
2020-08-28 15:10 ` Masami Hiramatsu
2020-08-28 15:18 ` peterz
2020-08-28 16:01 ` Masami Hiramatsu
2020-08-28 12:30 ` [PATCH v4 21/23] asm-generic/atomic: Add try_cmpxchg() fallbacks Masami Hiramatsu
2020-08-28 12:30 ` [PATCH v4 22/23] freelist: Lock less freelist Masami Hiramatsu
2020-08-28 12:30 ` [PATCH v4 23/23] kprobes: Replace rp->free_instance with freelist Masami Hiramatsu
2020-08-28 12:37 ` [PATCH v4 00/23] kprobes: Unify kretprobe trampoline handlers and make kretprobe lockless Masami Hiramatsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20200829033726.68547b37624d3510ebc33ab1@kernel.org \
--to=mhiramat@kernel.org \
--cc=Eddy_Wu@trendmicro.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=cameron@moodycamel.com \
--cc=davem@davemloft.net \
--cc=linux-arch@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=naveen.n.rao@linux.ibm.com \
--cc=oleg@redhat.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=will@kernel.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).