linux-trace-users.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: ahmadkhorrami <ahmadkhorrami@ut.ac.ir>
To: Milian Wolff <milian.wolff@kdab.com>
Cc: Jiri Olsa <jolsa@redhat.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Linux-trace Users <linux-trace-users@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-trace-users-owner@vger.kernel.org,
	Jin Yao <yao.jin@linux.intel.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Andi Kleen <ak@linux.intel.com>
Subject: Re: Wrong Perf Backtraces
Date: Tue, 31 Mar 2020 19:35:55 +0430	[thread overview]
Message-ID: <f7460b4bbedbeeb61d8fedf32c1213e9@ut.ac.ir> (raw)
In-Reply-To: <fe8e797932c0cb6d1b8db2ee9b991ed0@ut.ac.ir>

And it seems that the bogus backtraces constitute only a small portion 
of the whole log. This seems to be good news.

On 2020-03-31 19:32, ahmadkhorrami wrote:

> Hi Milian,
> Thanks for the detailed answer. Well, the bug you mentioned is bad 
> news. Because I sample using uppp. Perhaps this leads to these weird 
> traces. Is this a purely software bug?
> 
> On 2020-03-31 19:14, Milian Wolff wrote:
> 
> On Dienstag, 31. März 2020 15:39:18 CEST ahmadkhorrami wrote:
> 
> But the addresses do not match. Do you confirm this as a bug in
> libdwarf,...?
> 
> So I will ignore addresses without a matching symbol. But they do not
> seem reliable!
> 
> Could you tell me the name of the library that generates the raw
> addresses, so that I can try to debug it?
> This is a platform specific question. There are multiple ways to unwind 
> a
> stack. If you are on x86 then by default the .eh_frame section is 
> available
> which holds the information necessary for unwinding. It doesn't depend 
> on
> debug symbols, that's only used for symbolization and inline-frame 
> resolution
> as Jiri indicated.
> 
> That said, in the context of perf, there are multiple scenarios that 
> can lead
> to broken unwinding:
> 
> a) perf record --call-graph dwarf: unwinding can overflow the stack 
> copy
> associated with every sample, so the upper end of the stack will be 
> broken
> 
> b) perf record --call-graph $any: when you are sampling on a precise 
> event,
> such as cycles:P which is the default afaik, then on Intel with PEBS 
> e.g. the
> stack copy may be "wrong". See e.g. https://lkml.org/lkml/2018/11/6/257 
> and
> the overall thread. This is not solved yet afaik and after my initial 
> attempt
> at workarounding this issue I stopped looking into it and instead opted 
> for
> explicitly sampling on the non-precise events when I record call 
> graphs... You
> could try that too: do you see the issue when you run e.g.:
> 
> `perf record --call-graph dwarf -e cycles`
> 
> This should take the non-precise version for sampling but then at least 
> the
> call stacks are correct. I.e. you trade the accuracy of the instruction
> pointer to which a sample points with reduced call stack breakage.
> 
> c) bugs :)

  reply	other threads:[~2020-03-31 15:06 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <157597d74ff17f781d9de7e7e3defd13@ut.ac.ir>
2020-03-22 20:24 ` Wrong Perf Backtraces ahmadkhorrami
2020-03-23  0:34   ` Steven Rostedt
     [not found]     ` <21b3df4080709f193d62b159887e2a83@ut.ac.ir>
2020-03-23  8:49       ` Jiri Olsa
2020-03-23 10:03         ` ahmadkhorrami
2020-03-25 15:18           ` ahmadkhorrami
2020-03-25 15:46             ` Jiri Olsa
2020-03-25 18:54               ` ahmadkhorrami
2020-03-25 18:58               ` Arnaldo Carvalho de Melo
2020-03-25 19:10                 ` ahmadkhorrami
2020-03-25 19:28                   ` Arnaldo Carvalho de Melo
2020-03-25 20:01                     ` ahmadkhorrami
2020-03-25 20:39                       ` Jiri Olsa
2020-03-25 21:02                         ` Jiri Olsa
2020-03-25 21:09                           ` Steven Rostedt
2020-03-25 21:37                             ` ahmadkhorrami
2020-03-25 21:46                               ` Jiri Olsa
2020-03-25 22:21                                 ` ahmadkhorrami
2020-03-25 23:09                                   ` ahmadkhorrami
2020-03-26  9:59                                     ` Jiri Olsa
2020-03-26 13:20                                       ` ahmadkhorrami
2020-03-26 15:39                                         ` Jiri Olsa
2020-03-26 18:19                                           ` ahmadkhorrami
2020-03-26 18:21                                             ` ahmadkhorrami
2020-03-27  9:20                                             ` Jiri Olsa
2020-03-27 10:59                                               ` ahmadkhorrami
2020-03-27 11:04                                                 ` ahmadkhorrami
2020-03-27 12:10                                                   ` Milian Wolff
2020-03-27 12:58                                                     ` ahmadkhorrami
2020-03-27 13:25                                                       ` Milian Wolff
2020-03-27 13:33                                                         ` ahmadkhorrami
2020-03-27 18:43                                                   ` ahmadkhorrami
2020-03-27 22:37                                                     ` Jiri Olsa
2020-03-27 23:12                                                       ` ahmadkhorrami
2020-03-28 23:34                                                         ` Jiri Olsa
2020-03-29  0:43                                                           ` ahmadkhorrami
2020-03-29  1:16                                                             ` ahmadkhorrami
2020-03-29 11:19                                                               ` Jiri Olsa
2020-03-29 11:52                                                                 ` ahmadkhorrami
2020-03-29 12:08                                                                   ` Jiri Olsa
2020-03-29 12:39                                                                     ` ahmadkhorrami
2020-03-29 13:50                                                                       ` Milian Wolff
2020-03-29 14:23                                                                         ` ahmadkhorrami
2020-03-29 19:20                                                                         ` Jiri Olsa
2020-03-30  6:09                                                                           ` Milian Wolff
2020-03-30 13:07                                                                             ` Jiri Olsa
2020-03-30 13:49                                                                               ` ahmadkhorrami
2020-03-30 19:05                                                                                 ` ahmadkhorrami
2020-03-30 21:05                                                                                   ` debuginfod-based dwarf downloading, was " Frank Ch. Eigler
2020-03-31  9:26                                                                                     ` Jiri Olsa
2020-03-31 14:00                                                                                       ` Frank Ch. Eigler
2020-03-31  4:43                                                                                   ` ahmadkhorrami
2020-03-31  9:30                                                                                     ` Jiri Olsa
2020-03-31 11:53                                                                                       ` ahmadkhorrami
2020-03-31 12:43                                                                                   ` ahmadkhorrami
2020-03-31 13:20                                                                                     ` Jiri Olsa
2020-03-31 13:39                                                                                       ` ahmadkhorrami
2020-03-31 14:44                                                                                         ` Milian Wolff
2020-03-31 15:02                                                                                           ` ahmadkhorrami
2020-03-31 15:05                                                                                             ` ahmadkhorrami [this message]
2020-03-31 15:29                                                                                             ` Milian Wolff
2020-03-31 16:10                                                                                               ` Arnaldo Carvalho de Melo
2020-03-31 19:20                                                                                                 ` ahmadkhorrami
2020-03-31 19:17                                                                                               ` ahmadkhorrami
2020-03-31 20:57                                                                                               ` ahmadkhorrami
2020-04-04  1:01                                                                                                 ` ahmadkhorrami
2020-04-11 16:42                                                                                                   ` ahmadkhorrami
2020-04-11 21:04                                                                                                     ` ahmadkhorrami

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=f7460b4bbedbeeb61d8fedf32c1213e9@ut.ac.ir \
    --to=ahmadkhorrami@ut.ac.ir \
    --cc=acme@redhat.com \
    --cc=ak@linux.intel.com \
    --cc=jolsa@redhat.com \
    --cc=linux-trace-users-owner@vger.kernel.org \
    --cc=linux-trace-users@vger.kernel.org \
    --cc=milian.wolff@kdab.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=yao.jin@linux.intel.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).