From: ahmadkhorrami <ahmadkhorrami@ut.ac.ir>
To: Milian Wolff <milian.wolff@kdab.com>
Cc: Jiri Olsa <jolsa@redhat.com>,
Steven Rostedt <rostedt@goodmis.org>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Linux-trace Users <linux-trace-users@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
linux-trace-users-owner@vger.kernel.org,
Jin Yao <yao.jin@linux.intel.com>,
Namhyung Kim <namhyung@kernel.org>,
Andi Kleen <ak@linux.intel.com>
Subject: Re: Wrong Perf Backtraces
Date: Tue, 31 Mar 2020 19:35:55 +0430 [thread overview]
Message-ID: <f7460b4bbedbeeb61d8fedf32c1213e9@ut.ac.ir> (raw)
In-Reply-To: <fe8e797932c0cb6d1b8db2ee9b991ed0@ut.ac.ir>
And it seems that the bogus backtraces constitute only a small portion
of the whole log. This seems to be good news.
On 2020-03-31 19:32, ahmadkhorrami wrote:
> Hi Milian,
> Thanks for the detailed answer. Well, the bug you mentioned is bad
> news. Because I sample using uppp. Perhaps this leads to these weird
> traces. Is this a purely software bug?
>
> On 2020-03-31 19:14, Milian Wolff wrote:
>
> On Dienstag, 31. März 2020 15:39:18 CEST ahmadkhorrami wrote:
>
> But the addresses do not match. Do you confirm this as a bug in
> libdwarf,...?
>
> So I will ignore addresses without a matching symbol. But they do not
> seem reliable!
>
> Could you tell me the name of the library that generates the raw
> addresses, so that I can try to debug it?
> This is a platform specific question. There are multiple ways to unwind
> a
> stack. If you are on x86 then by default the .eh_frame section is
> available
> which holds the information necessary for unwinding. It doesn't depend
> on
> debug symbols, that's only used for symbolization and inline-frame
> resolution
> as Jiri indicated.
>
> That said, in the context of perf, there are multiple scenarios that
> can lead
> to broken unwinding:
>
> a) perf record --call-graph dwarf: unwinding can overflow the stack
> copy
> associated with every sample, so the upper end of the stack will be
> broken
>
> b) perf record --call-graph $any: when you are sampling on a precise
> event,
> such as cycles:P which is the default afaik, then on Intel with PEBS
> e.g. the
> stack copy may be "wrong". See e.g. https://lkml.org/lkml/2018/11/6/257
> and
> the overall thread. This is not solved yet afaik and after my initial
> attempt
> at workarounding this issue I stopped looking into it and instead opted
> for
> explicitly sampling on the non-precise events when I record call
> graphs... You
> could try that too: do you see the issue when you run e.g.:
>
> `perf record --call-graph dwarf -e cycles`
>
> This should take the non-precise version for sampling but then at least
> the
> call stacks are correct. I.e. you trade the accuracy of the instruction
> pointer to which a sample points with reduced call stack breakage.
>
> c) bugs :)
next prev parent reply other threads:[~2020-03-31 15:06 UTC|newest]
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <157597d74ff17f781d9de7e7e3defd13@ut.ac.ir>
2020-03-22 20:24 ` Wrong Perf Backtraces ahmadkhorrami
2020-03-23 0:34 ` Steven Rostedt
[not found] ` <21b3df4080709f193d62b159887e2a83@ut.ac.ir>
2020-03-23 8:49 ` Jiri Olsa
2020-03-23 10:03 ` ahmadkhorrami
2020-03-25 15:18 ` ahmadkhorrami
2020-03-25 15:46 ` Jiri Olsa
2020-03-25 18:54 ` ahmadkhorrami
2020-03-25 18:58 ` Arnaldo Carvalho de Melo
2020-03-25 19:10 ` ahmadkhorrami
2020-03-25 19:28 ` Arnaldo Carvalho de Melo
2020-03-25 20:01 ` ahmadkhorrami
2020-03-25 20:39 ` Jiri Olsa
2020-03-25 21:02 ` Jiri Olsa
2020-03-25 21:09 ` Steven Rostedt
2020-03-25 21:37 ` ahmadkhorrami
2020-03-25 21:46 ` Jiri Olsa
2020-03-25 22:21 ` ahmadkhorrami
2020-03-25 23:09 ` ahmadkhorrami
2020-03-26 9:59 ` Jiri Olsa
2020-03-26 13:20 ` ahmadkhorrami
2020-03-26 15:39 ` Jiri Olsa
2020-03-26 18:19 ` ahmadkhorrami
2020-03-26 18:21 ` ahmadkhorrami
2020-03-27 9:20 ` Jiri Olsa
2020-03-27 10:59 ` ahmadkhorrami
2020-03-27 11:04 ` ahmadkhorrami
2020-03-27 12:10 ` Milian Wolff
2020-03-27 12:58 ` ahmadkhorrami
2020-03-27 13:25 ` Milian Wolff
2020-03-27 13:33 ` ahmadkhorrami
2020-03-27 18:43 ` ahmadkhorrami
2020-03-27 22:37 ` Jiri Olsa
2020-03-27 23:12 ` ahmadkhorrami
2020-03-28 23:34 ` Jiri Olsa
2020-03-29 0:43 ` ahmadkhorrami
2020-03-29 1:16 ` ahmadkhorrami
2020-03-29 11:19 ` Jiri Olsa
2020-03-29 11:52 ` ahmadkhorrami
2020-03-29 12:08 ` Jiri Olsa
2020-03-29 12:39 ` ahmadkhorrami
2020-03-29 13:50 ` Milian Wolff
2020-03-29 14:23 ` ahmadkhorrami
2020-03-29 19:20 ` Jiri Olsa
2020-03-30 6:09 ` Milian Wolff
2020-03-30 13:07 ` Jiri Olsa
2020-03-30 13:49 ` ahmadkhorrami
2020-03-30 19:05 ` ahmadkhorrami
2020-03-30 21:05 ` debuginfod-based dwarf downloading, was " Frank Ch. Eigler
2020-03-31 9:26 ` Jiri Olsa
2020-03-31 14:00 ` Frank Ch. Eigler
2020-03-31 4:43 ` ahmadkhorrami
2020-03-31 9:30 ` Jiri Olsa
2020-03-31 11:53 ` ahmadkhorrami
2020-03-31 12:43 ` ahmadkhorrami
2020-03-31 13:20 ` Jiri Olsa
2020-03-31 13:39 ` ahmadkhorrami
2020-03-31 14:44 ` Milian Wolff
2020-03-31 15:02 ` ahmadkhorrami
2020-03-31 15:05 ` ahmadkhorrami [this message]
2020-03-31 15:29 ` Milian Wolff
2020-03-31 16:10 ` Arnaldo Carvalho de Melo
2020-03-31 19:20 ` ahmadkhorrami
2020-03-31 19:17 ` ahmadkhorrami
2020-03-31 20:57 ` ahmadkhorrami
2020-04-04 1:01 ` ahmadkhorrami
2020-04-11 16:42 ` ahmadkhorrami
2020-04-11 21:04 ` ahmadkhorrami
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f7460b4bbedbeeb61d8fedf32c1213e9@ut.ac.ir \
--to=ahmadkhorrami@ut.ac.ir \
--cc=acme@redhat.com \
--cc=ak@linux.intel.com \
--cc=jolsa@redhat.com \
--cc=linux-trace-users-owner@vger.kernel.org \
--cc=linux-trace-users@vger.kernel.org \
--cc=milian.wolff@kdab.com \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=yao.jin@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).