From: Masami Hiramatsu <mhiramat@kernel.org>
To: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
Steven Rostedt <rostedt@goodmis.org>,
Ingo Molnar <mingo@kernel.org>, X86 ML <x86@kernel.org>,
Daniel Xu <dxu@dxuuu.xyz>,
linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
kuba@kernel.org, mingo@redhat.com, ast@kernel.org,
tglx@linutronix.de, kernel-team@fb.com, yhs@fb.com,
Josh Poimboeuf <jpoimboe@redhat.com>,
linux-ia64@vger.kernel.org,
Abhishek Sagar <sagar.abhishek@gmail.com>
Subject: Re: [PATCH -tip v4 10/12] x86/kprobes: Push a fake return address at kretprobe_trampoline
Date: Fri, 26 Mar 2021 03:05:03 +0900 [thread overview]
Message-ID: <20210326030503.7fa72da34e25ad35cf5ed3de@kernel.org> (raw)
In-Reply-To: <20210324104058.7c06aaeb0408e24db6ba46f8@kernel.org>
On Wed, 24 Mar 2021 10:40:58 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:
> On Tue, 23 Mar 2021 23:30:07 +0100
> Peter Zijlstra <peterz@infradead.org> wrote:
>
> > On Mon, Mar 22, 2021 at 03:41:40PM +0900, Masami Hiramatsu wrote:
> > > ".global kretprobe_trampoline\n"
> > > ".type kretprobe_trampoline, @function\n"
> > > "kretprobe_trampoline:\n"
> > > #ifdef CONFIG_X86_64
> >
> > So what happens if we get an NMI here? That is, after the RET but before
> > the push? Then our IP points into the trampoline but we've not done that
> > push yet.
>
> Not only NMI, but also interrupts can happen. There is no cli/sti here.
>
> Anyway, thanks for pointing!
> I think in UNWIND_HINT_TYPE_REGS and UNWIND_HINT_TYPE_REGS_PARTIAL cases
> ORC unwinder also has to check the state->ip and if it is kretprobe_trampoline,
> it should be recovered.
> What about this?
Hmm, this seems to intoduce another issue on stacktrace from kprobes.
<...>-137 [003] d.Z. 17.250714: p_full_proxy_read_5: (full_proxy_read+0x5/0x80)
<...>-137 [003] d.Z. 17.250737: <stack trace>
=> kprobe_trace_func+0x1d0/0x2c0
=> kprobe_dispatcher+0x39/0x60
=> aggr_pre_handler+0x4f/0x90
=> kprobe_int3_handler+0x152/0x1a0
=> exc_int3+0x47/0x140
=> asm_exc_int3+0x31/0x40
=> 0
=> 0
=> 0
=> 0
=> 0
=> 0
=> 0
Let me check...
Thanks,
>
> diff --git a/arch/x86/include/asm/unwind.h b/arch/x86/include/asm/unwind.h
> index 332aa6174b10..36d3971c0a2c 100644
> --- a/arch/x86/include/asm/unwind.h
> +++ b/arch/x86/include/asm/unwind.h
> @@ -101,6 +101,15 @@ void unwind_module_init(struct module *mod, void *orc_ip, size_t orc_ip_size,
> void *orc, size_t orc_size) {}
> #endif
>
> +static inline
> +unsigned long unwind_recover_kretprobe(struct unwind_state *state,
> + unsigned long addr, unsigned long *addr_p)
> +{
> + return is_kretprobe_trampoline(addr) ?
> + kretprobe_find_ret_addr(state->task, addr_p, &state->kr_cur) :
> + addr;
> +}
> +
> /* Recover the return address modified by instrumentation (e.g. kretprobe) */
> static inline
> unsigned long unwind_recover_ret_addr(struct unwind_state *state,
> @@ -110,10 +119,7 @@ unsigned long unwind_recover_ret_addr(struct unwind_state *state,
>
> ret = ftrace_graph_ret_addr(state->task, &state->graph_idx,
> addr, addr_p);
> - if (is_kretprobe_trampoline(ret))
> - ret = kretprobe_find_ret_addr(state->task, addr_p,
> - &state->kr_cur);
> - return ret;
> + return unwind_recover_kretprobe(state, ret, addr_p);
> }
>
> /*
> diff --git a/arch/x86/kernel/unwind_orc.c b/arch/x86/kernel/unwind_orc.c
> index 839a0698342a..cb59aeca6a4a 100644
> --- a/arch/x86/kernel/unwind_orc.c
> +++ b/arch/x86/kernel/unwind_orc.c
> @@ -549,7 +549,15 @@ bool unwind_next_frame(struct unwind_state *state)
> (void *)orig_ip);
> goto err;
> }
> -
> + /*
> + * There is a small chance to interrupt at the entry of
> + * kretprobe_trampoline where the ORC info doesn't exist.
> + * That point is right after the RET to kretprobe_trampoline
> + * which was modified return address. So the @addr_p must
> + * be right before the regs->sp.
> + */
> + state->ip = unwind_recover_kretprobe(state, state->ip,
> + state->sp - sizeof(unsigned long));
> state->regs = (struct pt_regs *)sp;
> state->prev_regs = NULL;
> state->full_regs = true;
> @@ -562,6 +570,9 @@ bool unwind_next_frame(struct unwind_state *state)
> (void *)orig_ip);
> goto err;
> }
> + /* See UNWIND_HINT_TYPE_REGS case comment. */
> + state->ip = unwind_recover_kretprobe(state, state->ip,
> + state->sp - sizeof(unsigned long));
>
> if (state->full_regs)
> state->prev_regs = state->regs;
>
>
> --
> Masami Hiramatsu <mhiramat@kernel.org>
--
Masami Hiramatsu <mhiramat@kernel.org>
next prev parent reply other threads:[~2021-03-25 18:06 UTC|newest]
Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-22 6:39 [PATCH -tip v4 00/12] kprobes: Fix stacktrace with kretprobes on x86 Masami Hiramatsu
2021-03-22 6:39 ` [PATCH -tip v4 01/12] ia64: kprobes: Fix to pass correct trampoline address to the handler Masami Hiramatsu
2021-03-22 6:40 ` [PATCH -tip v4 02/12] kprobes: treewide: Replace arch_deref_entry_point() with dereference_function_descriptor() Masami Hiramatsu
2021-03-22 6:40 ` [PATCH -tip v4 03/12] kprobes: treewide: Remove trampoline_address from kretprobe_trampoline_handler() Masami Hiramatsu
2021-03-22 6:40 ` [PATCH -tip v4 04/12] kprobes: Add kretprobe_find_ret_addr() for searching return address Masami Hiramatsu
2021-03-22 6:40 ` [PATCH -tip v4 05/12] x86/kprobes: Add UNWIND_HINT_FUNC on kretprobe_trampoline code Masami Hiramatsu
2021-03-22 6:40 ` [PATCH -tip v4 06/12] ARC: Add instruction_pointer_set() API Masami Hiramatsu
2021-03-22 6:41 ` [PATCH -tip v4 07/12] ia64: " Masami Hiramatsu
2021-03-22 6:41 ` [PATCH -tip v4 08/12] arm: kprobes: Make a space for regs->ARM_pc at kretprobe_trampoline Masami Hiramatsu
2021-03-23 0:04 ` Masami Hiramatsu
2021-03-22 6:41 ` [PATCH -tip v4 09/12] kprobes: Setup instruction pointer in __kretprobe_trampoline_handler Masami Hiramatsu
2021-03-22 6:41 ` [PATCH -tip v4 10/12] x86/kprobes: Push a fake return address at kretprobe_trampoline Masami Hiramatsu
2021-03-23 22:30 ` Peter Zijlstra
2021-03-24 1:40 ` Masami Hiramatsu
2021-03-24 16:01 ` Josh Poimboeuf
2021-03-24 23:47 ` Masami Hiramatsu
2021-03-25 0:26 ` Steven Rostedt
2021-03-25 3:25 ` Masami Hiramatsu
2021-03-25 18:05 ` Masami Hiramatsu [this message]
2021-03-26 12:03 ` Masami Hiramatsu
2021-03-26 14:20 ` Steven Rostedt
2021-03-30 4:58 ` Josh Poimboeuf
2021-03-22 6:41 ` [PATCH -tip v4 11/12] x86/unwind: Recover kretprobe trampoline entry Masami Hiramatsu
2021-03-22 6:42 ` [PATCH -tip v4 12/12] tracing: Show kretprobe unknown indicator only for kretprobe_trampoline Masami Hiramatsu
2021-03-22 15:11 ` Steven Rostedt
2021-03-22 23:56 ` Masami Hiramatsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210326030503.7fa72da34e25ad35cf5ed3de@kernel.org \
--to=mhiramat@kernel.org \
--cc=ast@kernel.org \
--cc=bpf@vger.kernel.org \
--cc=dxu@dxuuu.xyz \
--cc=jpoimboe@redhat.com \
--cc=kernel-team@fb.com \
--cc=kuba@kernel.org \
--cc=linux-ia64@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=mingo@redhat.com \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=sagar.abhishek@gmail.com \
--cc=tglx@linutronix.de \
--cc=x86@kernel.org \
--cc=yhs@fb.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).