From: Aleksa Sarai <cyphar@cyphar.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>,
Anil S Keshavamurthy <anil.s.keshavamurthy@intel.com>,
"David S. Miller" <davem@davemloft.net>,
Masami Hiramatsu <mhiramat@kernel.org>,
Jonathan Corbet <corbet@lwn.net>,
Peter Zijlstra <peterz@infradead.org>,
Ingo Molnar <mingo@redhat.com>,
Arnaldo Carvalho de Melo <acme@kernel.org>,
Alexander Shishkin <alexander.shishkin@linux.intel.com>,
Jiri Olsa <jolsa@redhat.com>, Namhyung Kim <namhyung@kernel.org>,
Shuah Khan <shuah@kernel.org>,
Alexei Starovoitov <ast@kernel.org>,
Daniel Borkmann <daniel@iogearbox.net>,
Brendan Gregg <bgregg@netflix.com>,
Christian Brauner <christian@brauner.io>,
Aleksa Sarai <asarai@suse.de>,
netdev@vger.kernel.org, linux-doc@vger.kernel.org,
linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
Josh Poimboeuf <jpoimboe@redhat.com>
Subject: Re: [PATCH v3 1/2] kretprobe: produce sane stack traces
Date: Fri, 2 Nov 2018 16:05:09 +1100 [thread overview]
Message-ID: <20181102050509.tw3dhvj5urudvtjl@yavin> (raw)
In-Reply-To: <20181101204720.6ed3fe37@vmware.local.home>
[-- Attachment #1: Type: text/plain, Size: 2288 bytes --]
On 2018-11-01, Steven Rostedt <rostedt@goodmis.org> wrote:
> On Thu, 1 Nov 2018 19:35:50 +1100
> Aleksa Sarai <cyphar@cyphar.com> wrote:
> > @@ -1834,6 +1853,11 @@ static int pre_handler_kretprobe(struct kprobe *p, struct pt_regs *regs)
> > ri->rp = rp;
> > ri->task = current;
> >
> > + trace.entries = &ri->entry.entries[0];
> > + trace.max_entries = KRETPROBE_TRACE_SIZE;
> > + save_stack_trace_regs(regs, &trace);
> > + ri->entry.nr_entries = trace.nr_entries;
> > +
>
> So basically your saving a copy of the stack trace for each probe, for
> something that may not ever be used?
>
> This adds quite a lot of overhead for something not used often.
>
> I think the real answer is to fix kretprobes (and I just checked, the
> return call of function graph tracer stack trace doesn't go past the
> return_to_handler function either. It's just that a stack trace was
> never done on the return side).
>
> The more I look at this patch, the less I like it.
That's more than fair.
There's also an issue that Masami noted -- nested kretprobes don't give
you the full stack trace with this solution since the stack was already
overwritten. I think that the only real option is to fix the unwinder to
work in a kretprobe context -- which is what I'm looking at now.
Unfortunately, I'm having a lot of trouble understanding how the current
ftrace hooking works -- ORC has a couple of ftrace hooks that seem
reasonable on the surface but I don't understand (for instance) how
HAVE_FUNCTION_GRAPH_RET_ADDR_PTR *actually* works. Though your comment
appears to indicate that it doesn't work for stack traces?
For kretprobes I think it would be fairly easy to reconstruct what
landed you into a kretprobe_trampoline by walking the set of
kretprobe_instances (since all new ones are added to the head, you can
get the real return address in-order).
But I still have to figure out what is actually stopping the
save_stack_trace() unwinder that isn't stopping the show_stacks()
unwinder (though the show_stacks() code is more ... liberal with the
degree of certainty it has about the unwind).
Am I barking up the wrong tree?
--
Aleksa Sarai
Senior Software Engineer (Containers)
SUSE Linux GmbH
<https://www.cyphar.com/>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2018-11-02 5:05 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-11-01 8:35 [PATCH v3 0/2] kretprobe: produce sane stack traces Aleksa Sarai
2018-11-01 8:35 ` [PATCH v3 1/2] " Aleksa Sarai
2018-11-01 15:20 ` Masami Hiramatsu
2018-11-01 21:13 ` Aleksa Sarai
2018-11-02 3:04 ` Masami Hiramatsu
2018-11-02 4:37 ` Aleksa Sarai
2018-11-03 12:47 ` Masami Hiramatsu
2018-11-02 0:47 ` Steven Rostedt
2018-11-02 5:05 ` Aleksa Sarai [this message]
2018-11-02 6:59 ` Aleksa Sarai
2018-11-02 13:16 ` Steven Rostedt
2018-11-02 15:43 ` Josh Poimboeuf
2018-11-02 16:13 ` Steven Rostedt
2018-11-03 13:00 ` Masami Hiramatsu
2018-11-03 13:13 ` Steven Rostedt
2018-11-03 16:34 ` Masami Hiramatsu
2018-11-03 17:30 ` Steven Rostedt
2018-11-03 17:33 ` Steven Rostedt
2018-11-04 2:25 ` Masami Hiramatsu
2018-11-03 7:02 ` Aleksa Sarai
2018-11-04 11:59 ` Aleksa Sarai
2018-11-06 22:15 ` Steven Rostedt
2018-11-08 7:46 ` Aleksa Sarai
2018-11-08 8:04 ` Aleksa Sarai
2018-11-08 14:44 ` Josh Poimboeuf
2018-11-09 7:26 ` Masami Hiramatsu
2018-11-09 15:10 ` Aleksa Sarai
2018-11-09 7:15 ` Masami Hiramatsu
2018-11-09 15:06 ` Aleksa Sarai
2018-11-10 15:31 ` Masami Hiramatsu
2018-11-12 10:38 ` Aleksa Sarai
2018-11-03 13:23 ` Masami Hiramatsu
2018-11-02 7:58 ` Aleksa Sarai
2018-11-02 4:01 ` kbuild test robot
2018-11-01 8:35 ` [PATCH v3 2/2] trace: remove kretprobed checks Aleksa Sarai
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20181102050509.tw3dhvj5urudvtjl@yavin \
--to=cyphar@cyphar.com \
--cc=acme@kernel.org \
--cc=alexander.shishkin@linux.intel.com \
--cc=anil.s.keshavamurthy@intel.com \
--cc=asarai@suse.de \
--cc=ast@kernel.org \
--cc=bgregg@netflix.com \
--cc=christian@brauner.io \
--cc=corbet@lwn.net \
--cc=daniel@iogearbox.net \
--cc=davem@davemloft.net \
--cc=jolsa@redhat.com \
--cc=jpoimboe@redhat.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=mhiramat@kernel.org \
--cc=mingo@redhat.com \
--cc=namhyung@kernel.org \
--cc=naveen.n.rao@linux.vnet.ibm.com \
--cc=netdev@vger.kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=shuah@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).