From: Milian Wolff <milian.wolff@kdab.com>
To: Jiri Olsa <jolsa@redhat.com>, ahmadkhorrami <ahmadkhorrami@ut.ac.ir>
Cc: Steven Rostedt <rostedt@goodmis.org>,
Arnaldo Carvalho de Melo <acme@redhat.com>,
Linux-trace Users <linux-trace-users@vger.kernel.org>,
Peter Zijlstra <peterz@infradead.org>,
linux-trace-users-owner@vger.kernel.org,
Jin Yao <yao.jin@linux.intel.com>,
Namhyung Kim <namhyung@kernel.org>,
Andi Kleen <ak@linux.intel.com>
Subject: Re: Wrong Perf Backtraces
Date: Tue, 31 Mar 2020 16:44:18 +0200
Message-ID: <8573002.CDJkKcVGEf@agathebauer> (raw)
In-Reply-To: <08bf52eda6e699ee6b3070c75eac9123@ut.ac.ir>
[-- Attachment #1: Type: text/plain, Size: 1947 bytes --]
On Dienstag, 31. März 2020 15:39:18 CEST ahmadkhorrami wrote:
> But the addresses do not match. Do you confirm this as a bug in
> libdwarf,...?
>
> So I will ignore addresses without a matching symbol. But they do not
> seem reliable!
>
> Could you tell me the name of the library that generates the raw
> addresses, so that I can try to debug it?
This is a platform specific question. There are multiple ways to unwind a
stack. If you are on x86 then by default the .eh_frame section is available
which holds the information necessary for unwinding. It doesn't depend on
debug symbols, that's only used for symbolization and inline-frame resolution
as Jiri indicated.
That said, in the context of perf, there are multiple scenarios that can lead
to broken unwinding:
a) perf record --call-graph dwarf: unwinding can overflow the stack copy
associated with every sample, so the upper end of the stack will be broken
b) perf record --call-graph $any: when you are sampling on a precise event,
such as cycles:P which is the default afaik, then on Intel with PEBS e.g. the
stack copy may be "wrong". See e.g. https://lkml.org/lkml/2018/11/6/257 and
the overall thread. This is not solved yet afaik and after my initial attempt
at workarounding this issue I stopped looking into it and instead opted for
explicitly sampling on the non-precise events when I record call graphs... You
could try that too: do you see the issue when you run e.g.:
`perf record --call-graph dwarf -e cycles`
This should take the non-precise version for sampling but then at least the
call stacks are correct. I.e. you trade the accuracy of the instruction
pointer to which a sample points with reduced call stack breakage.
c) bugs :)
--
Milian Wolff | milian.wolff@kdab.com | Senior Software Engineer
KDAB (Deutschland) GmbH, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt, C++ and OpenGL Experts
[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 3826 bytes --]
next prev parent reply index
Thread overview: 67+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <157597d74ff17f781d9de7e7e3defd13@ut.ac.ir>
2020-03-22 20:24 ` ahmadkhorrami
2020-03-23 0:34 ` Steven Rostedt
[not found] ` <21b3df4080709f193d62b159887e2a83@ut.ac.ir>
2020-03-23 8:49 ` Jiri Olsa
2020-03-23 10:03 ` ahmadkhorrami
2020-03-25 15:18 ` ahmadkhorrami
2020-03-25 15:46 ` Jiri Olsa
2020-03-25 18:54 ` ahmadkhorrami
2020-03-25 18:58 ` Arnaldo Carvalho de Melo
2020-03-25 19:10 ` ahmadkhorrami
2020-03-25 19:28 ` Arnaldo Carvalho de Melo
2020-03-25 20:01 ` ahmadkhorrami
2020-03-25 20:39 ` Jiri Olsa
2020-03-25 21:02 ` Jiri Olsa
2020-03-25 21:09 ` Steven Rostedt
2020-03-25 21:37 ` ahmadkhorrami
2020-03-25 21:46 ` Jiri Olsa
2020-03-25 22:21 ` ahmadkhorrami
2020-03-25 23:09 ` ahmadkhorrami
2020-03-26 9:59 ` Jiri Olsa
2020-03-26 13:20 ` ahmadkhorrami
2020-03-26 15:39 ` Jiri Olsa
2020-03-26 18:19 ` ahmadkhorrami
2020-03-26 18:21 ` ahmadkhorrami
2020-03-27 9:20 ` Jiri Olsa
2020-03-27 10:59 ` ahmadkhorrami
2020-03-27 11:04 ` ahmadkhorrami
2020-03-27 12:10 ` Milian Wolff
2020-03-27 12:58 ` ahmadkhorrami
2020-03-27 13:25 ` Milian Wolff
2020-03-27 13:33 ` ahmadkhorrami
2020-03-27 18:43 ` ahmadkhorrami
2020-03-27 22:37 ` Jiri Olsa
2020-03-27 23:12 ` ahmadkhorrami
2020-03-28 23:34 ` Jiri Olsa
2020-03-29 0:43 ` ahmadkhorrami
2020-03-29 1:16 ` ahmadkhorrami
2020-03-29 11:19 ` Jiri Olsa
2020-03-29 11:52 ` ahmadkhorrami
2020-03-29 12:08 ` Jiri Olsa
2020-03-29 12:39 ` ahmadkhorrami
2020-03-29 13:50 ` Milian Wolff
2020-03-29 14:23 ` ahmadkhorrami
2020-03-29 19:20 ` Jiri Olsa
2020-03-30 6:09 ` Milian Wolff
2020-03-30 13:07 ` Jiri Olsa
2020-03-30 13:49 ` ahmadkhorrami
2020-03-30 19:05 ` ahmadkhorrami
2020-03-30 21:05 ` debuginfod-based dwarf downloading, was " Frank Ch. Eigler
2020-03-31 9:26 ` Jiri Olsa
2020-03-31 14:00 ` Frank Ch. Eigler
2020-03-31 4:43 ` ahmadkhorrami
2020-03-31 9:30 ` Jiri Olsa
2020-03-31 11:53 ` ahmadkhorrami
2020-03-31 12:43 ` ahmadkhorrami
2020-03-31 13:20 ` Jiri Olsa
2020-03-31 13:39 ` ahmadkhorrami
2020-03-31 14:44 ` Milian Wolff [this message]
2020-03-31 15:02 ` ahmadkhorrami
2020-03-31 15:05 ` ahmadkhorrami
2020-03-31 15:29 ` Milian Wolff
2020-03-31 16:10 ` Arnaldo Carvalho de Melo
2020-03-31 19:20 ` ahmadkhorrami
2020-03-31 19:17 ` ahmadkhorrami
2020-03-31 20:57 ` ahmadkhorrami
2020-04-04 1:01 ` ahmadkhorrami
2020-04-11 16:42 ` ahmadkhorrami
2020-04-11 21:04 ` ahmadkhorrami
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=8573002.CDJkKcVGEf@agathebauer \
--to=milian.wolff@kdab.com \
--cc=acme@redhat.com \
--cc=ahmadkhorrami@ut.ac.ir \
--cc=ak@linux.intel.com \
--cc=jolsa@redhat.com \
--cc=linux-trace-users-owner@vger.kernel.org \
--cc=linux-trace-users@vger.kernel.org \
--cc=namhyung@kernel.org \
--cc=peterz@infradead.org \
--cc=rostedt@goodmis.org \
--cc=yao.jin@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Linux-Trace-Users Archive on lore.kernel.org
Archives are clonable:
git clone --mirror https://lore.kernel.org/linux-trace-users/0 linux-trace-users/git/0.git
# If you have public-inbox 1.1+ installed, you may
# initialize and index your mirror using the following commands:
public-inbox-init -V2 linux-trace-users linux-trace-users/ https://lore.kernel.org/linux-trace-users \
linux-trace-users@vger.kernel.org
public-inbox-index linux-trace-users
Example config snippet for mirrors
Newsgroup available over NNTP:
nntp://nntp.lore.kernel.org/org.kernel.vger.linux-trace-users
AGPL code for this site: git clone https://public-inbox.org/public-inbox.git