From: Steven Rostedt <rostedt@goodmis.org>
To: Aleksa Sarai <cyphar@cyphar.com>
Cc: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>,
Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
"David S. Miller" <davem@davemloft.net>,
Masami Hiramatsu <mhiramat@kernel.org>,
Jonathan Corbet <corbet@lwn.net>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
Shuah Khan <shuah@kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Brendan Gregg <bgregg@netflix.com>,
Christian Brauner <christian@brauner.io>,
Aleksa Sarai <asarai@suse.de>,
netdev@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v3 1/2] kretprobe: produce sane stack traces
Date: Tue, 6 Nov 2018 17:15:01 -0500 [thread overview]
Message-ID: <20181106171501.59ccabbc@gandalf.local.home> (raw)
In-Reply-To: <20181104115913.74l4yzecisvtt2j5@yavin>
On Sun, 4 Nov 2018 22:59:13 +1100
Aleksa Sarai <cyphar@cyphar.com> wrote:
> The same issue is present in __save_stack_trace
> (arch/x86/kernel/stacktrace.c). This is likely the only reason that --
> as Steven said -- stacktraces wouldn't work with ftrace-graph (and thus
> with the refactor both of you are discussing).
By the way, I was playing with the the orc unwinder and stack traces
from the function graph tracer return code, and got it working with the
below patch. Caution, that patch also has a stack trace hardcoded in
the return path of the function graph tracer, so you don't want to run
function graph tracing without filtering.
You can apply the patch and do:
# cd /sys/kernel/debug/tracing
# echo schedule > set_ftrace_filter
# echo function_graph > current_tracer
# cat trace
3) | schedule() {
rcu_preempt-10 [003] .... 91.160297: <stack trace>
=> ftrace_return_to_handler
=> return_to_handler
=> schedule_timeout
=> rcu_gp_kthread
=> kthread
=> ret_from_fork
3) # 4009.085 us | }
3) | schedule() {
kworker/1:0-17 [001] .... 91.163288: <stack trace>
=> ftrace_return_to_handler
=> return_to_handler
=> worker_thread
=> kthread
=> ret_from_fork
1) # 7000.070 us | }
1) | schedule() {
rcu_preempt-10 [003] .... 91.164311: <stack trace>
=> ftrace_return_to_handler
=> return_to_handler
=> schedule_timeout
=> rcu_gp_kthread
=> kthread
=> ret_from_fork
3) # 4006.540 us | }
Where just adding the stack trace without the other code, these traces
ended at "return_to_handler".
This patch is not for inclusion, it was just a test to see how to make
this work.
-- Steve
diff --git a/arch/x86/kernel/ftrace_64.S b/arch/x86/kernel/ftrace_64.S
index 91b2cff4b79a..4bcd646ae1f4 100644
--- a/arch/x86/kernel/ftrace_64.S
+++ b/arch/x86/kernel/ftrace_64.S
@@ -320,13 +320,15 @@ ENTRY(ftrace_graph_caller)
ENDPROC(ftrace_graph_caller)
ENTRY(return_to_handler)
- UNWIND_HINT_EMPTY
- subq $24, %rsp
+ subq $8, %rsp
+ UNWIND_HINT_FUNC
+ subq $16, %rsp
/* Save the return values */
movq %rax, (%rsp)
movq %rdx, 8(%rsp)
movq %rbp, %rdi
+ leaq 16(%rsp), %rsi
call ftrace_return_to_handler
diff --git a/kernel/trace/trace_functions_graph.c b/kernel/trace/trace_functions_graph.c
index 169b3c44ee97..aaeca73218cc 100644
--- a/kernel/trace/trace_functions_graph.c
+++ b/kernel/trace/trace_functions_graph.c
@@ -242,13 +242,16 @@ ftrace_pop_return_trace(struct ftrace_graph_ret *trace, unsigned long *ret,
trace->calltime = current->ret_stack[index].calltime;
trace->overrun = atomic_read(¤t->trace_overrun);
trace->depth = index;
+
+ trace_dump_stack(0);
}
/*
* Send the trace to the ring-buffer.
* @return the original return address.
*/
-unsigned long ftrace_return_to_handler(unsigned long frame_pointer)
+unsigned long ftrace_return_to_handler(unsigned long frame_pointer,
+ unsigned long *ptr)
{
struct ftrace_graph_ret trace;
unsigned long ret;
@@ -257,6 +260,8 @@ unsigned long ftrace_return_to_handler(unsigned long frame_pointer)
trace.rettime = trace_clock_local();
barrier();
current->curr_ret_stack--;
+ *ptr = ret;
+
/*
* The curr_ret_stack can be less than -1 only if it was
* filtered out and it's about to return from the function.
next prev parent reply other threads:[~2018-11-06 22:15 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-01 8:35 [PATCH v3 0/2] kretprobe: produce sane stack traces Aleksa Sarai
2018-11-01 8:35 ` [PATCH v3 1/2] " Aleksa Sarai
2018-11-01 15:20 ` Masami Hiramatsu
2018-11-01 21:13 ` Aleksa Sarai
2018-11-02 3:04 ` Masami Hiramatsu
2018-11-02 4:37 ` Aleksa Sarai
2018-11-03 12:47 ` Masami Hiramatsu
2018-11-02 0:47 ` Steven Rostedt
2018-11-02 5:05 ` Aleksa Sarai
2018-11-02 6:59 ` Aleksa Sarai
2018-11-02 13:16 ` Steven Rostedt
2018-11-02 15:43 ` Josh Poimboeuf
2018-11-02 16:13 ` Steven Rostedt
2018-11-03 13:00 ` Masami Hiramatsu
2018-11-03 13:13 ` Steven Rostedt
2018-11-03 16:34 ` Masami Hiramatsu
2018-11-03 17:30 ` Steven Rostedt
2018-11-03 17:33 ` Steven Rostedt
2018-11-04 2:25 ` Masami Hiramatsu
2018-11-03 7:02 ` Aleksa Sarai
2018-11-04 11:59 ` Aleksa Sarai
2018-11-06 22:15 ` Steven Rostedt [this message]
2018-11-08 7:46 ` Aleksa Sarai
2018-11-08 8:04 ` Aleksa Sarai
2018-11-08 14:44 ` Josh Poimboeuf
2018-11-09 7:26 ` Masami Hiramatsu
2018-11-09 15:10 ` Aleksa Sarai
2018-11-09 7:15 ` Masami Hiramatsu
2018-11-09 15:06 ` Aleksa Sarai
2018-11-10 15:31 ` Masami Hiramatsu
2018-11-12 10:38 ` Aleksa Sarai
2018-11-03 13:23 ` Masami Hiramatsu
2018-11-02 7:58 ` Aleksa Sarai
2018-11-02 4:01 ` kbuild test robot
2018-11-01 8:35 ` [PATCH v3 2/2] trace: remove kretprobed checks Aleksa Sarai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181106171501.59ccabbc@gandalf.local.home \
--to=rostedt@goodmis.org \
--cc=acme@kernel.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=asarai@suse.de \
--cc=ast@kernel.org \
--cc=bgregg@netflix.com \
--cc=christian@brauner.io \
--cc=corbet@lwn.net \
--cc=cyphar@cyphar.com \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=jolsa@redhat.com \
--cc=jpoimboe@redhat.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=naveen.n.rao@linux.vnet.ibm.com \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).