From: Masami Hiramatsu <mhiramat@kernel.org>
To: Mark Rutland <mark.rutland@arm.com>
Cc: Steven Rostedt <rostedt@goodmis.org>,
"Naveen N . Rao" <naveen.n.rao@linux.vnet.ibm.com>,
Ananth N Mavinakayanahalli <ananth@linux.ibm.com>,
Ingo Molnar <mingo@kernel.org>,
linux-kernel@vger.kernel.org, Sven Schnelle <svens@linux.ibm.com>,
Catalin Marinas <catalin.marinas@arm.com>,
Will Deacon <will@kernel.org>,
Russell King <linux@armlinux.org.uk>,
Nathan Chancellor <nathan@kernel.org>,
Nick Desaulniers <ndesaulniers@google.com>,
linux-arm-kernel@lists.infradead.org
Subject: Re: [PATCH 5/8] arm64: Recover kretprobe modified return address in stacktrace
Date: Thu, 14 Oct 2021 18:57:20 +0900 [thread overview]
Message-ID: <20211014185720.67262334328b383e082d9db4@kernel.org> (raw)
In-Reply-To: <20211013101351.GB3187@C02TD0UTHF1T.local>
On Wed, 13 Oct 2021 11:13:51 +0100
Mark Rutland <mark.rutland@arm.com> wrote:
> On Fri, Oct 08, 2021 at 09:28:58PM +0900, Masami Hiramatsu wrote:
> > Since the kretprobe replaces the function return address with
> > the kretprobe_trampoline on the stack, stack unwinder shows it
> > instead of the correct return address.
> >
> > This checks whether the next return address is the
> > __kretprobe_trampoline(), and if so, try to find the correct
> > return address from the kretprobe instance list.
> >
> > With this fix, now arm64 can enable
> > CONFIG_ARCH_CORRECT_STACKTRACE_ON_KRETPROBE, and pass the
> > kprobe self tests.
> >
> > Signed-off-by: Masami Hiramatsu <mhiramat@kernel.org>
> > ---
> > arch/arm64/Kconfig | 1 +
> > arch/arm64/include/asm/stacktrace.h | 2 ++
> > arch/arm64/kernel/stacktrace.c | 3 +++
> > 3 files changed, 6 insertions(+)
> >
> > diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
> > index 5c7ae4c3954b..edde5171ffb2 100644
> > --- a/arch/arm64/Kconfig
> > +++ b/arch/arm64/Kconfig
> > @@ -11,6 +11,7 @@ config ARM64
> > select ACPI_PPTT if ACPI
> > select ARCH_HAS_DEBUG_WX
> > select ARCH_BINFMT_ELF_STATE
> > + select ARCH_CORRECT_STACKTRACE_ON_KRETPROBE
> > select ARCH_ENABLE_HUGEPAGE_MIGRATION if HUGETLB_PAGE && MIGRATION
> > select ARCH_ENABLE_MEMORY_HOTPLUG
> > select ARCH_ENABLE_MEMORY_HOTREMOVE
> > diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h
> > index 8aebc00c1718..8f997a602651 100644
> > --- a/arch/arm64/include/asm/stacktrace.h
> > +++ b/arch/arm64/include/asm/stacktrace.h
> > @@ -9,6 +9,7 @@
> > #include <linux/sched.h>
> > #include <linux/sched/task_stack.h>
> > #include <linux/types.h>
> > +#include <linux/llist.h>
> >
> > #include <asm/memory.h>
> > #include <asm/ptrace.h>
> > @@ -59,6 +60,7 @@ struct stackframe {
> > #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> > int graph;
> > #endif
> > + struct llist_node *kr_cur;
>
> As with the fgraph bits above, please make this depedn on the relevant
> Kconfig, i.e.
>
> | #ifdef CONFIG_KRETPROBES
> | struct llist_node *kr_cur;
> | #endif
>
> > };
OK.
> >
> > extern int unwind_frame(struct task_struct *tsk, struct stackframe *frame);
> > diff --git a/arch/arm64/kernel/stacktrace.c b/arch/arm64/kernel/stacktrace.c
> > index 8982a2b78acf..f1eef5745542 100644
> > --- a/arch/arm64/kernel/stacktrace.c
> > +++ b/arch/arm64/kernel/stacktrace.c
> > @@ -129,6 +129,8 @@ int notrace unwind_frame(struct task_struct *tsk, struct stackframe *frame)
> > frame->pc = ret_stack->ret;
> > }
> > #endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> > + if (is_kretprobe_trampoline(frame->pc))
> > + frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur);
>
> Please ifdef this, like the CONFIG_FUNCTION_GRAPH_TRACER bits above.
> i.e.
>
> | #ifdef CONFIG_KRETPROBES
> | if (is_kretprobe_trampoline(frame->pc))
> | frame->pc = kretprobe_find_ret_addr(tsk, (void *)frame->fp, &frame->kr_cur);
> | #endif
OK.
>
> What does kretprobe_find_ret_addr() do when it can't find the original
> address? I couldn't spot it in this series or in mainline.
If it couldn't find, it returns NULL.
Hmm, should we check it and keep frame->pc if the return value is NULL?
(anyway, it must not happen. If it happens, that task can not continue to run.)
> As a future thing, I'd like to be able to have monotonicity and
> completeness checks as part of the unwind, i.e. checking that we consume
> the kretprobe address *in-order*, and can identify whether we've skipped
> any, so that we can identify when unwinding has gone wrong. Does it do
> that today?
Good question. No today, but is easy to do since we have the loop cursor
(frame->kr_cur).
unsigned long kretprobe_find_ret_addr(struct task_struct *tsk, void *fp,
struct llist_node **cur)
{
...
do {
ret = __kretprobe_find_ret_addr(tsk, cur);
if (!ret)
break;
ri = container_of(*cur, struct kretprobe_instance, llist);
} while (ri->fp != fp);
...
Change this loop to;
ri = container_of(*cur, struct kretprobe_instance, llist);
prev_fp = ri->fp;
do {
ret = __kretprobe_find_ret_addr(tsk, cur);
if (WARN_ON_ONCE(!ret))
return ERR_PTR(-ENOENT);
ri = container_of(*cur, struct kretprobe_instance, llist);
} while (ri->fp == prev_fp);
if (ri->fp != fp)
return ERR_PTR(-EILSEQ);
Then, we can detect that wrong sequence from stacktrace side.
>
> It'd be nice if it could signal failure reliably (without causing a
> BUG() or similar), e.g. by returning an error code.
As above, -EILSEQ is OK?
>
> > frame->pc = ptrauth_strip_insn_pac(frame->pc);
> >
> > @@ -224,6 +226,7 @@ noinline notrace void arch_stack_walk(stack_trace_consume_fn consume_entry,
> > {
> > struct stackframe frame;
> >
> > + memset(&frame, 0, sizeof(frame));
>
> Please initialise stackframe::kr_cur in start_backtrace, where we
> initialize all the other fields in struct stackframe, i.e. just after
> the CONFIG_FUNCTION_GRAPH_TRACER bit, have:
>
> | #ifdef CONFIG_KRETPROBES
> | frame->kr_cur = NULL;
> | #endif
OK, let me update it.
Thank you!
>
> Thanks,
> Mark.
>
> > if (regs)
> > start_backtrace(&frame, regs->regs[29], regs->pc);
> > else if (task == current)
> >
--
Masami Hiramatsu <mhiramat@kernel.org>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2021-10-14 9:59 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-10-08 12:28 [PATCH 0/8] kprobes: Make KUnit and add stacktrace on kretprobe tests Masami Hiramatsu
2021-10-08 12:28 ` [PATCH 1/8] kprobes: convert tests to kunit Masami Hiramatsu
2021-10-08 12:28 ` [PATCH 2/8] kprobes: Add a test case for stacktrace from kretprobe handler Masami Hiramatsu
2021-10-08 12:28 ` [PATCH 3/8] arm64: kprobes: Record frame pointer with kretprobe instance Masami Hiramatsu
2021-10-13 8:14 ` Will Deacon
2021-10-13 10:01 ` Mark Rutland
2021-10-14 8:04 ` Masami Hiramatsu
2021-10-14 9:13 ` Mark Rutland
2021-10-14 10:01 ` Masami Hiramatsu
2021-10-14 10:27 ` Mark Rutland
2021-10-14 13:50 ` Masami Hiramatsu
2021-10-08 12:28 ` [PATCH 4/8] arm64: kprobes: Make a frame pointer on __kretprobe_trampoline Masami Hiramatsu
2021-10-13 8:14 ` Will Deacon
2021-10-08 12:28 ` [PATCH 5/8] arm64: Recover kretprobe modified return address in stacktrace Masami Hiramatsu
2021-10-13 8:14 ` Will Deacon
2021-10-14 8:05 ` Masami Hiramatsu
2021-10-13 10:13 ` Mark Rutland
2021-10-14 9:57 ` Masami Hiramatsu [this message]
2021-10-08 12:29 ` [PATCH 6/8] ARM: clang: Do not relay on lr register for stacktrace Masami Hiramatsu
2021-10-11 18:45 ` Nick Desaulniers
2021-10-12 14:18 ` Masami Hiramatsu
2021-10-13 19:54 ` Nick Desaulniers
2021-10-14 16:53 ` Russell King (Oracle)
2021-10-15 0:18 ` Masami Hiramatsu
2021-10-08 12:29 ` [PATCH 7/8] ARM: kprobes: Make a frame pointer on __kretprobe_trampoline Masami Hiramatsu
2021-10-11 19:06 ` Nick Desaulniers
2021-10-08 12:29 ` [PATCH 8/8] ARM: Recover kretprobe modified return address in stacktrace Masami Hiramatsu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20211014185720.67262334328b383e082d9db4@kernel.org \
--to=mhiramat@kernel.org \
--cc=ananth@linux.ibm.com \
--cc=catalin.marinas@arm.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux@armlinux.org.uk \
--cc=mark.rutland@arm.com \
--cc=mingo@kernel.org \
--cc=nathan@kernel.org \
--cc=naveen.n.rao@linux.vnet.ibm.com \
--cc=ndesaulniers@google.com \
--cc=rostedt@goodmis.org \
--cc=svens@linux.ibm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).