From: Steven Rostedt <rostedt@goodmis.org>
To: Josh Poimboeuf <jpoimboe@redhat.com>
Cc: Thomas Gleixner <tglx@linutronix.de>,
Ingo Molnar <mingo@kernel.org>, "H . Peter Anvin" <hpa@zytor.com>,
x86@kernel.org, linux-kernel@vger.kernel.org,
Andy Lutomirski <luto@amacapital.net>,
Linus Torvalds <torvalds@linux-foundation.org>,
Brian Gerst <brgerst@gmail.com>,
Kees Cook <keescook@chromium.org>,
Peter Zijlstra <peterz@infradead.org>,
Frederic Weisbecker <fweisbec@gmail.com>,
Byungchul Park <byungchul.park@lge.com>
Subject: Re: [PATCH 05/19] x86/dumpstack: fix function graph tracing stack dump reliability issues
Date: Fri, 29 Jul 2016 18:55:21 -0400 [thread overview]
Message-ID: <20160729185521.62a5cd2a@gandalf.local.home> (raw)
In-Reply-To: <ee70f40bd7773f2bc77b9ce55869163cce2aa9f2.1469136008.git.jpoimboe@redhat.com>
On Thu, 21 Jul 2016 16:21:42 -0500
Josh Poimboeuf <jpoimboe@redhat.com> wrote:
> When function graph tracing is enabled for a function, its return
> address on the stack is replaced with the address of an ftrace handler
> (return_to_handler). When dumping the stack of a task with graph
> tracing enabled, there are some subtle bugs:
>
> - The fake return_to_handler() address can be reported as reliable.
> Instead, because it's not the real caller, it should be considered
> unreliable.
I have some mixed emotions about this. First, it's not "fake", the
function *is* going to return to it, but you are right, that's not the
function that was called.
I do like to see these in the trace, because sometimes these functions
are an issue. But I guess I can live with them being marked as
"unreliable".
>
> - In print_context_stack(), the real caller's return address is always
> reported as reliable, even if the return_to_handler() address wasn't
> referred to by a frame pointer.
Hmm, if CONFIG_FRAME_POINTER is enabled, perhaps we should only call
the look up of ftrace_graph_ret_addr(). Hmm, playing with this, yeah,
we definitely should. It can report the wrong reliability.
Without doing the reliability check we can get out of sync with the
ret_stack. I have a patch to go on top of this patch below (hmm, it may
not apply fully, because I was using a different base tree than you).
>
> In addition to fixing these bugs, convert print_ftrace_graph_addr() to a
> more generic function which can be used outside of dump_trace()
> callbacks.
>
> Signed-off-by: Josh Poimboeuf <jpoimboe@redhat.com>
> ---
> arch/x86/include/asm/stacktrace.h | 13 ++++++++++
> arch/x86/kernel/dumpstack.c | 50 +++++++++++++++++----------------------
> 2 files changed, 35 insertions(+), 28 deletions(-)
>
> diff --git a/arch/x86/include/asm/stacktrace.h b/arch/x86/include/asm/stacktrace.h
> index 6f65995..5d3d258 100644
> --- a/arch/x86/include/asm/stacktrace.h
> +++ b/arch/x86/include/asm/stacktrace.h
> @@ -14,6 +14,19 @@ extern int kstack_depth_to_print;
> struct thread_info;
> struct stacktrace_ops;
>
> +#ifdef CONFIG_FUNCTION_GRAPH_TRACER
> +
> +unsigned long
> +ftrace_graph_ret_addr(struct task_struct *task, int *idx, unsigned long addr);
> +
> +#else
> +static inline unsigned long
> +ftrace_graph_ret_addr(struct task_struct *task, int *idx, unsigned long addr)
> +{
> + return addr;
> +}
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
> +
> typedef unsigned long (*walk_stack_t)(struct task_struct *task,
> unsigned long *stack,
> unsigned long bp,
> diff --git a/arch/x86/kernel/dumpstack.c b/arch/x86/kernel/dumpstack.c
> index 692eecae..0a8694b 100644
> --- a/arch/x86/kernel/dumpstack.c
> +++ b/arch/x86/kernel/dumpstack.c
> @@ -40,36 +40,25 @@ void printk_address(unsigned long address)
> }
>
> #ifdef CONFIG_FUNCTION_GRAPH_TRACER
> -static void
> -print_ftrace_graph_addr(unsigned long addr, void *data,
> - const struct stacktrace_ops *ops,
> - struct task_struct *task, int *graph)
> +unsigned long
> +ftrace_graph_ret_addr(struct task_struct *task, int *idx, unsigned long addr)
> {
> - unsigned long ret_addr;
> - int index;
> + int task_idx;
>
> if (addr != (unsigned long)return_to_handler)
> - return;
> + return addr;
>
> - index = task->curr_ret_stack;
> + task_idx = task->curr_ret_stack;
>
> - if (!task->ret_stack || index < *graph)
> - return;
> + if (!task->ret_stack || task_idx < *idx)
> + return addr;
>
> - index -= *graph;
> - ret_addr = task->ret_stack[index].ret;
> + task_idx -= *idx;
> + (*idx)++;
>
> - ops->address(data, ret_addr, 1);
> -
> - (*graph)++;
> + return task->ret_stack[task_idx].ret;
> }
> -#else
> -static inline void
> -print_ftrace_graph_addr(unsigned long addr, void *data,
> - const struct stacktrace_ops *ops,
> - struct task_struct *task, int *graph)
> -{ }
> -#endif
> +#endif /* CONFIG_FUNCTION_GRAPH_TRACER */
>
> /*
> * x86-64 can have up to three kernel stacks:
> @@ -108,18 +97,23 @@ print_context_stack(struct task_struct *task,
> stack = (unsigned long *)task_stack_page(task);
>
> while (valid_stack_ptr(task, stack, sizeof(*stack), end)) {
> - unsigned long addr;
> + unsigned long addr = *stack;
>
> addr = *stack;
> if (__kernel_text_address(addr)) {
> + int reliable = 0;
> + unsigned long real_addr;
> +
> if ((unsigned long) stack == bp + sizeof(long)) {
> - ops->address(data, addr, 1);
> + reliable = 1;
> frame = frame->next_frame;
> bp = (unsigned long) frame;
> - } else {
> - ops->address(data, addr, 0);
> }
> - print_ftrace_graph_addr(addr, data, ops, task, graph);
> +
> + real_addr = ftrace_graph_ret_addr(task, graph, addr);
> + if (addr != real_addr)
> + ops->address(data, addr, 0);
Note this changes behavior, as the original code had the ret_to_handler
first. This makes it second. (I fixed this below).
And that we should add a reliability check if CONFIG_FRAME_POINTER is
enabled.
> + ops->address(data, real_addr, reliable);
> }
> stack++;
> }
> @@ -142,11 +136,11 @@ print_context_stack_bp(struct task_struct *task,
> if (!__kernel_text_address(addr))
> break;
>
> + addr = ftrace_graph_ret_addr(task, graph, addr);
> if (ops->address(data, addr, 1))
> break;
> frame = frame->next_frame;
> ret_addr = &frame->return_address;
> - print_ftrace_graph_addr(addr, data, ops, task, graph);
This also changes the current code to print the return address as well.
> }
>
> return (unsigned long)frame;
Here's my patch that should be applied on top.
Maybe add a Signed-off-by: Steven Rostedt <rostedt@goodmis.org> along
with your SOB. But you should remain Author.
-- Steve
---
arch/x86/kernel/dumpstack.c | 16 ++++++++++------
1 file changed, 10 insertions(+), 6 deletions(-)
Index: linux-trace.git/arch/x86/kernel/dumpstack.c
===================================================================
--- linux-trace.git.orig/arch/x86/kernel/dumpstack.c 2016-07-29 17:17:10.995002677 -0400
+++ linux-trace.git/arch/x86/kernel/dumpstack.c 2016-07-29 18:50:53.497633797 -0400
@@ -90,10 +90,9 @@ print_context_stack(struct task_struct *
while (valid_stack_ptr(task, stack, sizeof(*stack), end)) {
unsigned long addr = *stack;
- addr = *stack;
if (__kernel_text_address(addr)) {
+ unsigned long real_addr = addr;
int reliable = 0;
- unsigned long real_addr;
if ((unsigned long) stack == bp + sizeof(long)) {
reliable = 1;
@@ -101,10 +100,12 @@ print_context_stack(struct task_struct *
bp = (unsigned long) frame;
}
- real_addr = ftrace_graph_ret_addr(task, graph, addr);
+ if (!IS_ENABLED(CONFIG_FRAME_POINTER) || reliable)
+ real_addr = ftrace_graph_ret_addr(task, graph, addr);
+
+ ops->address(data, real_addr, reliable);
if (addr != real_addr)
ops->address(data, addr, 0);
- ops->address(data, real_addr, reliable);
}
stack++;
}
@@ -123,13 +124,16 @@ print_context_stack_bp(struct task_struc
while (valid_stack_ptr(task, ret_addr, sizeof(*ret_addr), end)) {
unsigned long addr = *ret_addr;
+ unsigned long real_addr;
if (!__kernel_text_address(addr))
break;
- addr = ftrace_graph_ret_addr(task, graph, addr);
- if (ops->address(data, addr, 1))
+ real_addr = ftrace_graph_ret_addr(task, graph, addr);
+ if (ops->address(data, real_addr, 1))
break;
+ if (real_addr != addr)
+ ops->address(data, addr, 0);
frame = frame->next_frame;
ret_addr = &frame->return_address;
}
next prev parent reply other threads:[~2016-07-29 22:55 UTC|newest]
Thread overview: 91+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-07-21 21:21 [PATCH 00/19] x86/dumpstack: rewrite x86 stack dump code Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 01/19] x86/dumpstack: remove show_trace() Josh Poimboeuf
2016-07-21 21:49 ` Andy Lutomirski
2016-07-21 21:21 ` [PATCH 02/19] x86/dumpstack: add get_stack_pointer() and get_frame_pointer() Josh Poimboeuf
2016-07-21 21:53 ` Andy Lutomirski
2016-07-21 21:21 ` [PATCH 03/19] x86/dumpstack: remove unnecessary stack pointer arguments Josh Poimboeuf
2016-07-21 21:56 ` Andy Lutomirski
2016-07-22 1:41 ` Josh Poimboeuf
2016-07-22 2:29 ` Andy Lutomirski
2016-07-22 3:08 ` Brian Gerst
2016-07-21 21:21 ` [PATCH 04/19] x86/dumpstack: make printk_stack_address() more generally useful Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 05/19] x86/dumpstack: fix function graph tracing stack dump reliability issues Josh Poimboeuf
2016-07-29 22:55 ` Steven Rostedt [this message]
2016-07-30 0:50 ` Josh Poimboeuf
2016-07-30 2:20 ` Steven Rostedt
2016-07-30 13:51 ` Josh Poimboeuf
2016-08-01 14:28 ` Steven Rostedt
2016-08-01 15:36 ` Josh Poimboeuf
2016-08-02 21:00 ` Josh Poimboeuf
2016-08-02 21:16 ` Steven Rostedt
2016-08-02 22:13 ` Josh Poimboeuf
2016-08-02 23:16 ` Steven Rostedt
2016-08-03 1:56 ` Josh Poimboeuf
2016-08-03 2:30 ` Steven Rostedt
2016-08-03 2:50 ` Josh Poimboeuf
2016-08-03 2:59 ` Steven Rostedt
2016-08-03 3:12 ` Josh Poimboeuf
2016-08-03 3:18 ` Steven Rostedt
2016-08-03 3:21 ` Steven Rostedt
2016-08-03 3:31 ` Josh Poimboeuf
2016-08-03 3:45 ` Steven Rostedt
2016-08-03 14:13 ` Josh Poimboeuf
2016-08-03 3:30 ` Josh Poimboeuf
2016-08-01 15:59 ` Josh Poimboeuf
2016-08-01 16:05 ` Steven Rostedt
2016-08-01 16:19 ` Josh Poimboeuf
2016-08-01 16:24 ` Josh Poimboeuf
2016-08-01 16:56 ` Steven Rostedt
2016-07-21 21:21 ` [PATCH 06/19] x86/dumpstack: remove extra brackets around "EOE" Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 07/19] x86/dumpstack: add IRQ_USABLE_STACK_SIZE define Josh Poimboeuf
2016-07-21 22:01 ` Andy Lutomirski
2016-07-22 1:48 ` Josh Poimboeuf
2016-07-22 8:24 ` Ingo Molnar
2016-07-21 21:21 ` [PATCH 08/19] x86/dumpstack: don't disable preemption in show_stack_log_lvl() and dump_trace() Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 09/19] x86/dumpstack: simplify in_exception_stack() Josh Poimboeuf
2016-07-21 22:05 ` Andy Lutomirski
2016-07-21 21:21 ` [PATCH 10/19] x86/dumpstack: add get_stack_info() interface Josh Poimboeuf
2016-07-22 23:26 ` Andy Lutomirski
2016-07-22 23:52 ` Andy Lutomirski
2016-07-23 13:09 ` Josh Poimboeuf
2016-07-22 23:54 ` Josh Poimboeuf
2016-07-23 0:15 ` Andy Lutomirski
2016-07-23 14:04 ` Josh Poimboeuf
2016-07-26 0:09 ` Andy Lutomirski
2016-07-26 16:26 ` Josh Poimboeuf
2016-07-26 17:51 ` Steven Rostedt
2016-07-26 18:56 ` Josh Poimboeuf
2016-07-26 20:59 ` Andy Lutomirski
2016-07-26 22:24 ` Josh Poimboeuf
2016-07-26 22:31 ` Steven Rostedt
2016-07-26 22:37 ` Andy Lutomirski
2016-07-26 16:47 ` Josh Poimboeuf
2016-07-26 17:49 ` Brian Gerst
2016-07-26 18:59 ` Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 11/19] x86/dumptrace: add new unwind interface and implementations Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 12/19] perf/x86: convert perf_callchain_kernel() to the new unwinder Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 13/19] x86/stacktrace: convert save_stack_trace_*() " Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 14/19] oprofile/x86: convert x86_backtrace() " Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 15/19] x86/dumpstack: convert show_trace_log_lvl() " Josh Poimboeuf
2016-07-21 21:49 ` Byungchul Park
2016-07-22 1:38 ` Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 16/19] x86/dumpstack: remove dump_trace() Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 17/19] x86/entry/dumpstack: encode pt_regs pointer in frame pointer Josh Poimboeuf
2016-07-21 22:27 ` Andy Lutomirski
2016-07-21 21:21 ` [PATCH 18/19] x86/dumpstack: print stack identifier on its own line Josh Poimboeuf
2016-07-21 21:21 ` [PATCH 19/19] x86/dumpstack: print any pt_regs found on the stack Josh Poimboeuf
2016-07-21 22:32 ` Andy Lutomirski
2016-07-22 3:30 ` Josh Poimboeuf
2016-07-22 5:13 ` Andy Lutomirski
2016-07-22 15:57 ` Josh Poimboeuf
2016-07-22 21:46 ` Andy Lutomirski
2016-07-22 22:20 ` Josh Poimboeuf
2016-07-22 23:18 ` Andy Lutomirski
2016-07-22 23:30 ` Josh Poimboeuf
2016-07-22 23:39 ` Andy Lutomirski
2016-07-23 0:00 ` Josh Poimboeuf
2016-07-23 0:22 ` [PATCH 00/19] x86/dumpstack: rewrite x86 stack dump code Linus Torvalds
2016-07-23 0:31 ` Andy Lutomirski
2016-07-23 5:35 ` Josh Poimboeuf
2016-07-23 5:39 ` Linus Torvalds
2016-07-23 12:53 ` Josh Poimboeuf
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160729185521.62a5cd2a@gandalf.local.home \
--to=rostedt@goodmis.org \
--cc=brgerst@gmail.com \
--cc=byungchul.park@lge.com \
--cc=fweisbec@gmail.com \
--cc=hpa@zytor.com \
--cc=jpoimboe@redhat.com \
--cc=keescook@chromium.org \
--cc=linux-kernel@vger.kernel.org \
--cc=luto@amacapital.net \
--cc=mingo@kernel.org \
--cc=peterz@infradead.org \
--cc=tglx@linutronix.de \
--cc=torvalds@linux-foundation.org \
--cc=x86@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).