All of lore.kernel.org
 help / color / mirror / Atom feed
From: Masami Hiramatsu <mhiramat@kernel.org>
To: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ingo Molnar <mingo@kernel.org>, X86 ML <x86@kernel.org>,
	Daniel Xu <dxu@dxuuu.xyz>,
	linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	kuba@kernel.org, mingo@redhat.com, ast@kernel.org,
	tglx@linutronix.de, kernel-team@fb.com, yhs@fb.com,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	linux-ia64@vger.kernel.org,
	Abhishek Sagar <sagar.abhishek@gmail.com>
Subject: Re: [PATCH -tip v4 10/12] x86/kprobes: Push a fake return address at kretprobe_trampoline
Date: Fri, 26 Mar 2021 21:03:49 +0900	[thread overview]
Message-ID: <20210326210349.22f6d34b229dd3a139a53686@kernel.org> (raw)
In-Reply-To: <20210326030503.7fa72da34e25ad35cf5ed3de@kernel.org>

On Fri, 26 Mar 2021 03:05:03 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Wed, 24 Mar 2021 10:40:58 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > On Tue, 23 Mar 2021 23:30:07 +0100
> > Peter Zijlstra <peterz@infradead.org> wrote:
> > 
> > > On Mon, Mar 22, 2021 at 03:41:40PM +0900, Masami Hiramatsu wrote:
> > > >  	".global kretprobe_trampoline\n"
> > > >  	".type kretprobe_trampoline, @function\n"
> > > >  	"kretprobe_trampoline:\n"
> > > >  #ifdef CONFIG_X86_64
> > > 
> > > So what happens if we get an NMI here? That is, after the RET but before
> > > the push? Then our IP points into the trampoline but we've not done that
> > > push yet.
> > 
> > Not only NMI, but also interrupts can happen. There is no cli/sti here.
> > 
> > Anyway, thanks for pointing!
> > I think in UNWIND_HINT_TYPE_REGS and UNWIND_HINT_TYPE_REGS_PARTIAL cases
> > ORC unwinder also has to check the state->ip and if it is kretprobe_trampoline,
> > it should be recovered.
> > What about this?
> 
> Hmm, this seems to intoduce another issue on stacktrace from kprobes.
> 
>            <...>-137     [003] d.Z.    17.250714: p_full_proxy_read_5: (full_proxy_read+0x5/0x80)
>            <...>-137     [003] d.Z.    17.250737: <stack trace>
>  => kprobe_trace_func+0x1d0/0x2c0
>  => kprobe_dispatcher+0x39/0x60
>  => aggr_pre_handler+0x4f/0x90
>  => kprobe_int3_handler+0x152/0x1a0
>  => exc_int3+0x47/0x140
>  => asm_exc_int3+0x31/0x40
>  => 0
>  => 0
>  => 0
>  => 0
>  => 0
>  => 0
>  => 0
> 
> Let me check...

I confirmed this is not related to this series, but occurs when I build kernels with different
configs without cleanup.

Once I build kernel with CONFIG_UNWIND_GUESS=y (for testing), and after that,
I build kernel again with CONFIG_UNWIND_ORC=y (but without make clean), this
happened. In this case, I guess ORC data might be corrupted?
When I cleanup and rebuild, the stacktrace seems correct.

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

WARNING: multiple messages have this Message-ID (diff)
From: Masami Hiramatsu <mhiramat@kernel.org>
To: Masami Hiramatsu <mhiramat@kernel.org>
Cc: Peter Zijlstra <peterz@infradead.org>,
	Steven Rostedt <rostedt@goodmis.org>,
	Ingo Molnar <mingo@kernel.org>, X86 ML <x86@kernel.org>,
	Daniel Xu <dxu@dxuuu.xyz>,
	linux-kernel@vger.kernel.org, bpf@vger.kernel.org,
	kuba@kernel.org, mingo@redhat.com, ast@kernel.org,
	tglx@linutronix.de, kernel-team@fb.com, yhs@fb.com,
	Josh Poimboeuf <jpoimboe@redhat.com>,
	linux-ia64@vger.kernel.org,
	Abhishek Sagar <sagar.abhishek@gmail.com>
Subject: Re: [PATCH -tip v4 10/12] x86/kprobes: Push a fake return address at kretprobe_trampoline
Date: Fri, 26 Mar 2021 12:03:49 +0000	[thread overview]
Message-ID: <20210326210349.22f6d34b229dd3a139a53686@kernel.org> (raw)
In-Reply-To: <20210326030503.7fa72da34e25ad35cf5ed3de@kernel.org>

On Fri, 26 Mar 2021 03:05:03 +0900
Masami Hiramatsu <mhiramat@kernel.org> wrote:

> On Wed, 24 Mar 2021 10:40:58 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
> 
> > On Tue, 23 Mar 2021 23:30:07 +0100
> > Peter Zijlstra <peterz@infradead.org> wrote:
> > 
> > > On Mon, Mar 22, 2021 at 03:41:40PM +0900, Masami Hiramatsu wrote:
> > > >  	".global kretprobe_trampoline\n"
> > > >  	".type kretprobe_trampoline, @function\n"
> > > >  	"kretprobe_trampoline:\n"
> > > >  #ifdef CONFIG_X86_64
> > > 
> > > So what happens if we get an NMI here? That is, after the RET but before
> > > the push? Then our IP points into the trampoline but we've not done that
> > > push yet.
> > 
> > Not only NMI, but also interrupts can happen. There is no cli/sti here.
> > 
> > Anyway, thanks for pointing!
> > I think in UNWIND_HINT_TYPE_REGS and UNWIND_HINT_TYPE_REGS_PARTIAL cases
> > ORC unwinder also has to check the state->ip and if it is kretprobe_trampoline,
> > it should be recovered.
> > What about this?
> 
> Hmm, this seems to intoduce another issue on stacktrace from kprobes.
> 
>            <...>-137     [003] d.Z.    17.250714: p_full_proxy_read_5: (full_proxy_read+0x5/0x80)
>            <...>-137     [003] d.Z.    17.250737: <stack trace>
>  => kprobe_trace_func+0x1d0/0x2c0
>  => kprobe_dispatcher+0x39/0x60
>  => aggr_pre_handler+0x4f/0x90
>  => kprobe_int3_handler+0x152/0x1a0
>  => exc_int3+0x47/0x140
>  => asm_exc_int3+0x31/0x40
>  => 0
>  => 0
>  => 0
>  => 0
>  => 0
>  => 0
>  => 0
> 
> Let me check...

I confirmed this is not related to this series, but occurs when I build kernels with different
configs without cleanup.

Once I build kernel with CONFIG_UNWIND_GUESS=y (for testing), and after that,
I build kernel again with CONFIG_UNWIND_ORC=y (but without make clean), this
happened. In this case, I guess ORC data might be corrupted?
When I cleanup and rebuild, the stacktrace seems correct.

Thank you,

-- 
Masami Hiramatsu <mhiramat@kernel.org>

  reply	other threads:[~2021-03-26 12:04 UTC|newest]

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-03-22  6:39 [PATCH -tip v4 00/12] kprobes: Fix stacktrace with kretprobes on x86 Masami Hiramatsu
2021-03-22  6:39 ` Masami Hiramatsu
2021-03-22  6:39 ` [PATCH -tip v4 01/12] ia64: kprobes: Fix to pass correct trampoline address to the handler Masami Hiramatsu
2021-03-22  6:39   ` Masami Hiramatsu
2021-03-22  6:40 ` [PATCH -tip v4 02/12] kprobes: treewide: Replace arch_deref_entry_point() with dereference_function_descriptor() Masami Hiramatsu
2021-03-22  6:40   ` [PATCH -tip v4 02/12] kprobes: treewide: Replace arch_deref_entry_point() with dereference_function_ Masami Hiramatsu
2021-03-22  6:40 ` [PATCH -tip v4 03/12] kprobes: treewide: Remove trampoline_address from kretprobe_trampoline_handler() Masami Hiramatsu
2021-03-22  6:40   ` [PATCH -tip v4 03/12] kprobes: treewide: Remove trampoline_address from kretprobe_trampoline_handler Masami Hiramatsu
2021-03-22  6:40 ` [PATCH -tip v4 04/12] kprobes: Add kretprobe_find_ret_addr() for searching return address Masami Hiramatsu
2021-03-22  6:40   ` Masami Hiramatsu
2021-03-22  6:40 ` [PATCH -tip v4 05/12] x86/kprobes: Add UNWIND_HINT_FUNC on kretprobe_trampoline code Masami Hiramatsu
2021-03-22  6:40   ` Masami Hiramatsu
2021-03-22  6:40 ` [PATCH -tip v4 06/12] ARC: Add instruction_pointer_set() API Masami Hiramatsu
2021-03-22  6:40   ` Masami Hiramatsu
2021-03-22  6:41 ` [PATCH -tip v4 07/12] ia64: " Masami Hiramatsu
2021-03-22  6:41   ` Masami Hiramatsu
2021-03-22  6:41 ` [PATCH -tip v4 08/12] arm: kprobes: Make a space for regs->ARM_pc at kretprobe_trampoline Masami Hiramatsu
2021-03-22  6:41   ` Masami Hiramatsu
2021-03-23  0:04   ` Masami Hiramatsu
2021-03-23  0:04     ` Masami Hiramatsu
2021-03-22  6:41 ` [PATCH -tip v4 09/12] kprobes: Setup instruction pointer in __kretprobe_trampoline_handler Masami Hiramatsu
2021-03-22  6:41   ` Masami Hiramatsu
2021-03-22  6:41 ` [PATCH -tip v4 10/12] x86/kprobes: Push a fake return address at kretprobe_trampoline Masami Hiramatsu
2021-03-22  6:41   ` Masami Hiramatsu
2021-03-23 22:30   ` Peter Zijlstra
2021-03-23 22:30     ` Peter Zijlstra
2021-03-24  1:40     ` Masami Hiramatsu
2021-03-24  1:40       ` Masami Hiramatsu
2021-03-24 16:01       ` Josh Poimboeuf
2021-03-24 16:01         ` Josh Poimboeuf
2021-03-24 23:47         ` Masami Hiramatsu
2021-03-24 23:47           ` Masami Hiramatsu
2021-03-25  0:26           ` Steven Rostedt
2021-03-25  0:26             ` Steven Rostedt
2021-03-25  3:25             ` Masami Hiramatsu
2021-03-25  3:25               ` Masami Hiramatsu
2021-03-25 18:05       ` Masami Hiramatsu
2021-03-25 18:05         ` Masami Hiramatsu
2021-03-26 12:03         ` Masami Hiramatsu [this message]
2021-03-26 12:03           ` Masami Hiramatsu
2021-03-26 14:20           ` Steven Rostedt
2021-03-26 14:20             ` Steven Rostedt
2021-03-30  4:58             ` Josh Poimboeuf
2021-03-30  4:58               ` Josh Poimboeuf
2021-03-22  6:41 ` [PATCH -tip v4 11/12] x86/unwind: Recover kretprobe trampoline entry Masami Hiramatsu
2021-03-22  6:41   ` Masami Hiramatsu
2021-03-22  6:42 ` [PATCH -tip v4 12/12] tracing: Show kretprobe unknown indicator only for kretprobe_trampoline Masami Hiramatsu
2021-03-22  6:42   ` Masami Hiramatsu
2021-03-22 15:11   ` Steven Rostedt
2021-03-22 15:11     ` Steven Rostedt
2021-03-22 23:56     ` Masami Hiramatsu
2021-03-22 23:56       ` Masami Hiramatsu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210326210349.22f6d34b229dd3a139a53686@kernel.org \
    --to=mhiramat@kernel.org \
    --cc=ast@kernel.org \
    --cc=bpf@vger.kernel.org \
    --cc=dxu@dxuuu.xyz \
    --cc=jpoimboe@redhat.com \
    --cc=kernel-team@fb.com \
    --cc=kuba@kernel.org \
    --cc=linux-ia64@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@kernel.org \
    --cc=mingo@redhat.com \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=sagar.abhishek@gmail.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    --cc=yhs@fb.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body. 
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.