From: Andrii Nakryiko <andrii.nakryiko@gmail.com>
To: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Steven Rostedt <rostedt@goodmis.org>,
Josh Poimboeuf <jpoimboe@redhat.com>,
Ingo Molnar <mingo@kernel.org>, X86 ML <x86@kernel.org>,
Daniel Xu <dxu@dxuuu.xyz>,
open list <linux-kernel@vger.kernel.org>,
bpf <bpf@vger.kernel.org>, Jakub Kicinski <kuba@kernel.org>,
Ingo Molnar <mingo@redhat.com>,
Alexei Starovoitov <ast@kernel.org>,
Thomas Gleixner <tglx@linutronix.de>,
Borislav Petkov <bp@alien8.de>,
Peter Zijlstra <peterz@infradead.org>,
Kernel Team <kernel-team@fb.com>, Yonghong Song <yhs@fb.com>,
linux-ia64@vger.kernel.org,
Abhishek Sagar <sagar.abhishek@gmail.com>,
Paul McKenney <paulmck@kernel.org>
Subject: Re: [PATCH -tip v11 00/27] kprobes: Fix stacktrace with kretprobes on x86
Date: Tue, 14 Sep 2021 15:55:40 -0700 [thread overview]
Message-ID: <CAEf4BzaAHhEgsyBV07Q37FkUSh9wypuJW6HqJ1jS8-XrjzKDmA@mail.gmail.com> (raw)
In-Reply-To: <163163030719.489837.2236069935502195491.stgit@devnote2>
On Tue, Sep 14, 2021 at 7:38 AM Masami Hiramatsu <mhiramat@kernel.org> wrote:
>
> Hello,
>
> This is the 11th version of the series to fix the stacktrace with kretprobe on x86.
>
> The previous version is here;
>
> https://lore.kernel.org/all/162756755600.301564.4957591913842010341.stgit@devnote2/
>
> This version is rebased on the latest tip/master branch and includes the kprobe cleanup
> series[1][2]. No code change.
>
> [1] https://lore.kernel.org/bpf/162748615977.59465.13262421617578791515.stgit@devnote2/
> [2] https://lore.kernel.org/linux-csky/20210727133426.2919710-1-punitagrawal@gmail.com/
>
>
> With this series, unwinder can unwind stack correctly from ftrace as below;
>
> # cd /sys/kernel/debug/tracing
> # echo > trace
> # echo 1 > options/sym-offset
> # echo r vfs_read >> kprobe_events
> # echo r full_proxy_read >> kprobe_events
> # echo traceoff:1 > events/kprobes/r_vfs_read_0/trigger
> # echo stacktrace:1 > events/kprobes/r_full_proxy_read_0/trigger
> # echo 1 > events/kprobes/enable
> # cat /sys/kernel/debug/kprobes/list
> ffffffff813bedf0 r full_proxy_read+0x0 [FTRACE]
> ffffffff812c13e0 r vfs_read+0x0 [FTRACE]
> # echo 0 > events/kprobes/enable
> # cat trace
> # tracer: nop
> #
> # entries-in-buffer/entries-written: 3/3 #P:8
> #
> # _-----=> irqs-off
> # / _----=> need-resched
> # | / _---=> hardirq/softirq
> # || / _--=> preempt-depth
> # ||| / _-=> migrate-disable
> # |||| / delay
> # TASK-PID CPU# ||||| TIMESTAMP FUNCTION
> # | | | ||||| | |
> cat-136 [000] ...1. 14.474966: r_full_proxy_read_0: (vfs_read+0x99/0x190 <- full_proxy_read)
> cat-136 [000] ...1. 14.474970: <stack trace>
> => kretprobe_trace_func+0x209/0x300
> => kretprobe_dispatcher+0x9d/0xb0
> => __kretprobe_trampoline_handler+0xd4/0x1b0
> => trampoline_handler+0x43/0x60
> => __kretprobe_trampoline+0x2a/0x50
> => vfs_read+0x99/0x190
> => ksys_read+0x68/0xe0
> => do_syscall_64+0x3b/0x90
> => entry_SYSCALL_64_after_hwframe+0x44/0xae
> cat-136 [000] ...1. 14.474971: r_vfs_read_0: (ksys_read+0x68/0xe0 <- vfs_read)
>
> This shows the double return probes (vfs_read() and full_proxy_read()) on the stack
> correctly unwinded. (vfs_read() returns to 'ksys_read+0x68' and full_proxy_read()
> returns to 'vfs_read+0x99')
>
> This also changes the kretprobe behavisor a bit, now the instraction pointer in
> the 'pt_regs' passed to kretprobe user handler is correctly set the real return
> address. So user handlers can get it via instruction_pointer() API, and can use
> stack_trace_save_regs().
>
> You can also get this series from
> git://git.kernel.org/pub/scm/linux/kernel/git/mhiramat/linux.git kprobes/kretprobe-stackfix-v11
>
>
> Thank you,
>
> ---
>
> Josh Poimboeuf (3):
> objtool: Add frame-pointer-specific function ignore
> objtool: Ignore unwind hints for ignored functions
> x86/kprobes: Add UNWIND_HINT_FUNC on kretprobe_trampoline()
>
> Masami Hiramatsu (19):
> kprobes: treewide: Cleanup the error messages for kprobes
> kprobes: Fix coding style issues
> kprobes: Use IS_ENABLED() instead of kprobes_built_in()
> kprobes: Add assertions for required lock
> kprobes: treewide: Use 'kprobe_opcode_t *' for the code address in get_optimized_kprobe()
> kprobes: Use bool type for functions which returns boolean value
> ia64: kprobes: Fix to pass correct trampoline address to the handler
> kprobes: treewide: Replace arch_deref_entry_point() with dereference_symbol_descriptor()
> kprobes: treewide: Remove trampoline_address from kretprobe_trampoline_handler()
> kprobes: treewide: Make it harder to refer kretprobe_trampoline directly
> kprobes: Add kretprobe_find_ret_addr() for searching return address
> ARC: Add instruction_pointer_set() API
> ia64: Add instruction_pointer_set() API
> arm: kprobes: Make space for instruction pointer on stack
> kprobes: Enable stacktrace from pt_regs in kretprobe handler
> x86/kprobes: Push a fake return address at kretprobe_trampoline
> x86/unwind: Recover kretprobe trampoline entry
> tracing: Show kretprobe unknown indicator only for kretprobe_trampoline
> x86/kprobes: Fixup return address in generic trampoline handler
>
> Punit Agrawal (5):
> kprobes: Do not use local variable when creating debugfs file
> kprobes: Use helper to parse boolean input from userspace
> kprobe: Simplify prepare_kprobe() by dropping redundant version
> csky: ftrace: Drop duplicate implementation of arch_check_ftrace_location()
> kprobes: Make arch_check_ftrace_location static
>
Re-tested with latest BPF selftests and retsnoop tool utilizing
kretprobe and capturing stack traces (broken without these changes).
Looks good, thank you!
Tested-by: Andrii Nakryiko <andriin@kernel.org>
>
> arch/arc/include/asm/kprobes.h | 2
> arch/arc/include/asm/ptrace.h | 5
> arch/arc/kernel/kprobes.c | 13 -
> arch/arm/probes/kprobes/core.c | 15 -
> arch/arm/probes/kprobes/opt-arm.c | 7
> arch/arm64/include/asm/kprobes.h | 2
> arch/arm64/kernel/probes/kprobes.c | 10
> arch/arm64/kernel/probes/kprobes_trampoline.S | 4
> arch/csky/include/asm/kprobes.h | 2
> arch/csky/kernel/probes/ftrace.c | 7
> arch/csky/kernel/probes/kprobes.c | 14 -
> arch/csky/kernel/probes/kprobes_trampoline.S | 4
> arch/ia64/include/asm/ptrace.h | 5
> arch/ia64/kernel/kprobes.c | 15 -
> arch/mips/kernel/kprobes.c | 26 +
> arch/parisc/kernel/kprobes.c | 6
> arch/powerpc/include/asm/kprobes.h | 2
> arch/powerpc/kernel/kprobes.c | 29 -
> arch/powerpc/kernel/optprobes.c | 8
> arch/powerpc/kernel/stacktrace.c | 2
> arch/riscv/include/asm/kprobes.h | 2
> arch/riscv/kernel/probes/kprobes.c | 15 -
> arch/riscv/kernel/probes/kprobes_trampoline.S | 4
> arch/s390/include/asm/kprobes.h | 2
> arch/s390/kernel/kprobes.c | 16 -
> arch/s390/kernel/stacktrace.c | 2
> arch/sh/include/asm/kprobes.h | 2
> arch/sh/kernel/kprobes.c | 12 -
> arch/sparc/include/asm/kprobes.h | 2
> arch/sparc/kernel/kprobes.c | 12 -
> arch/x86/include/asm/kprobes.h | 1
> arch/x86/include/asm/unwind.h | 23 +
> arch/x86/include/asm/unwind_hints.h | 5
> arch/x86/kernel/kprobes/core.c | 71 +++-
> arch/x86/kernel/kprobes/opt.c | 6
> arch/x86/kernel/unwind_frame.c | 3
> arch/x86/kernel/unwind_guess.c | 3
> arch/x86/kernel/unwind_orc.c | 21 +
> include/linux/kprobes.h | 113 ++++--
> include/linux/objtool.h | 12 +
> kernel/kprobes.c | 502 ++++++++++++++-----------
> kernel/trace/trace_kprobe.c | 2
> kernel/trace/trace_output.c | 17 -
> lib/error-inject.c | 3
> tools/include/linux/objtool.h | 12 +
> tools/objtool/check.c | 2
> 46 files changed, 607 insertions(+), 436 deletions(-)
>
> --
> Masami Hiramatsu (Linaro) <mhiramat@kernel.org>
next prev parent reply other threads:[~2021-09-14 22:55 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-14 14:38 [PATCH -tip v11 00/27] kprobes: Fix stacktrace with kretprobes on x86 Masami Hiramatsu
2021-09-14 14:38 ` [PATCH -tip v11 01/27] kprobes: Do not use local variable when creating debugfs file Masami Hiramatsu
2021-09-14 14:38 ` [PATCH -tip v11 02/27] kprobes: Use helper to parse boolean input from userspace Masami Hiramatsu
2021-09-14 14:38 ` [PATCH -tip v11 03/27] kprobe: Simplify prepare_kprobe() by dropping redundant version Masami Hiramatsu
2021-09-14 14:39 ` [PATCH -tip v11 04/27] csky: ftrace: Drop duplicate implementation of arch_check_ftrace_location() Masami Hiramatsu
2021-09-14 14:39 ` [PATCH -tip v11 05/27] kprobes: Make arch_check_ftrace_location static Masami Hiramatsu
2021-09-14 14:39 ` [PATCH -tip v11 06/27] kprobes: treewide: Cleanup the error messages for kprobes Masami Hiramatsu
2021-09-14 14:39 ` [PATCH -tip v11 07/27] kprobes: Fix coding style issues Masami Hiramatsu
2021-09-14 14:39 ` [PATCH -tip v11 08/27] kprobes: Use IS_ENABLED() instead of kprobes_built_in() Masami Hiramatsu
2021-09-14 14:39 ` [PATCH -tip v11 09/27] kprobes: Add assertions for required lock Masami Hiramatsu
2021-09-14 14:40 ` [PATCH -tip v11 10/27] kprobes: treewide: Use 'kprobe_opcode_t *' for the code address in get_optimized_kprobe() Masami Hiramatsu
2021-09-14 14:40 ` [PATCH -tip v11 11/27] kprobes: Use bool type for functions which returns boolean value Masami Hiramatsu
2021-09-14 14:40 ` [PATCH -tip v11 12/27] ia64: kprobes: Fix to pass correct trampoline address to the handler Masami Hiramatsu
2021-09-14 14:40 ` [PATCH -tip v11 13/27] kprobes: treewide: Replace arch_deref_entry_point() with dereference_symbol_descriptor() Masami Hiramatsu
2021-09-14 14:40 ` [PATCH -tip v11 14/27] kprobes: treewide: Remove trampoline_address from kretprobe_trampoline_handler() Masami Hiramatsu
2021-09-14 14:40 ` [PATCH -tip v11 15/27] kprobes: treewide: Make it harder to refer kretprobe_trampoline directly Masami Hiramatsu
2021-09-14 14:41 ` [PATCH -tip v11 16/27] kprobes: Add kretprobe_find_ret_addr() for searching return address Masami Hiramatsu
2021-09-14 14:41 ` [PATCH -tip v11 17/27] objtool: Add frame-pointer-specific function ignore Masami Hiramatsu
2021-09-14 14:41 ` [PATCH -tip v11 18/27] objtool: Ignore unwind hints for ignored functions Masami Hiramatsu
2021-09-14 14:41 ` [PATCH -tip v11 19/27] x86/kprobes: Add UNWIND_HINT_FUNC on kretprobe_trampoline() Masami Hiramatsu
2021-09-14 14:41 ` [PATCH -tip v11 20/27] ARC: Add instruction_pointer_set() API Masami Hiramatsu
2021-09-14 14:41 ` [PATCH -tip v11 21/27] ia64: " Masami Hiramatsu
2021-09-14 14:42 ` [PATCH -tip v11 22/27] arm: kprobes: Make space for instruction pointer on stack Masami Hiramatsu
2021-09-14 14:42 ` [PATCH -tip v11 23/27] kprobes: Enable stacktrace from pt_regs in kretprobe handler Masami Hiramatsu
2021-09-14 14:42 ` [PATCH -tip v11 24/27] x86/kprobes: Push a fake return address at kretprobe_trampoline Masami Hiramatsu
2021-09-14 14:42 ` [PATCH -tip v11 25/27] x86/unwind: Recover kretprobe trampoline entry Masami Hiramatsu
2021-09-14 14:42 ` [PATCH -tip v11 26/27] tracing: Show kretprobe unknown indicator only for kretprobe_trampoline Masami Hiramatsu
2021-09-14 14:42 ` [PATCH -tip v11 27/27] x86/kprobes: Fixup return address in generic trampoline handler Masami Hiramatsu
2021-09-14 22:55 ` Andrii Nakryiko [this message]
2021-09-29 2:24 ` [PATCH -tip v11 00/27] kprobes: Fix stacktrace with kretprobes on x86 Masami Hiramatsu
2021-09-30 18:17 ` Alexei Starovoitov
2021-09-30 19:34 ` Thomas Gleixner
2021-09-30 21:22 ` Steven Rostedt
2021-09-30 23:11 ` Thomas Gleixner
2021-09-30 23:27 ` Masami Hiramatsu
2021-09-30 23:37 ` Steven Rostedt
2021-10-01 0:35 ` Masami Hiramatsu
2021-09-30 23:54 ` Masami Hiramatsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=CAEf4BzaAHhEgsyBV07Q37FkUSh9wypuJW6HqJ1jS8-XrjzKDmA@mail.gmail.com \
--to=andrii.nakryiko@gmail.com \
--cc=ast@kernel.org \
--cc=bp@alien8.de \
--cc=bpf@vger.kernel.org \
--cc=dxu@dxuuu.xyz \
--cc=jpoimboe@redhat.com \
--cc=kernel-team@fb.com \
--cc=kuba@kernel.org \
--cc=linux-ia64@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mingo@kernel.org \
--cc=mingo@redhat.com \
--cc=paulmck@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=sagar.abhishek@gmail.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).