Re: Wrong Perf Backtraces

From: ahmadkhorrami <ahmadkhorrami@ut.ac.ir>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Milian Wolff <milian.wolff@kdab.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Linux-trace Users <linux-trace-users@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-trace-users-owner@vger.kernel.org,
	Jin Yao <yao.jin@linux.intel.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Andi Kleen <ak@linux.intel.com>
Subject: Re: Wrong Perf Backtraces
Date: Tue, 31 Mar 2020 09:13:28 +0430	[thread overview]
Message-ID: <c1a64287c45da58def62a45e44525f02@ut.ac.ir> (raw)
In-Reply-To: <8cb4f94b11fbac6ca3a44b924b6ee7b3@ut.ac.ir>

Hi,
I checked the reported address in libgtk. GDB could map it to the source 
files. Also, once, I read at Stackoverflow that perf gives up on 
printing the backtrace (truncates the output backtrace), whenever it 
detects a binary file without debug info. Could you describe more about 
this situation?
And dear Mr. Olsa, if these cases do not occur on your system, could you 
tell me how you installed the debug info for the binaries on your 
system, please?
Regards.

On 2020-03-30 23:35, ahmadkhorrami wrote:

> Hi,
> Sorry, I did not pay attention to the fact that we need debug info to 
> be able to decode the callchain. So, without complete debug info the 
> backtrace (either the symbols or the addresses) is bogus. But you say 
> that the backtrace portions containing binaries with full debug info 
> are OK. Am I right? Please confirm if this is the case.
> If so, these portions may be enough for my current usecase.
> Regards.
> 
> On 2020-03-30 18:19, ahmadkhorrami wrote:
> 
> Hi,
> thanks, but I still do not understand the cause of the problem. It 
> seems that it is purely a software bug and the callchain addresses seem 
> wrong,by themselves. Am I right?
> Regards.
> On 2020-03-30 17:37, Jiri Olsa wrote:
> 
> On Mon, Mar 30, 2020 at 08:09:53AM +0200, Milian Wolff wrote: On 
> Sonntag, 29. März 2020 21:20:10 CEST Jiri Olsa wrote: On Sun, Mar 29, 
> 2020 at 03:50:57PM +0200, Milian Wolff wrote: On Sonntag, 29. März 2020 
> 14:39:33 CEST ahmadkhorrami wrote: Thanks. I did both of your changes. 
> Perhaps some outputs are revised.
> But I still have repeated function calls and the script detects them.
> Here is one of them when the sampling period is 1000 events: <snip>
> 
> Here, we have two consecutive "7f91788120b5
> gtk_widget_propagate_state+0x195
> (/usr/lib/x86_64-linux-gnu/libgtk-3.so.0.2200.30) gtkwidget.c:0" lines,
> while "gtk_widget_propagate_state+0x195" is not recursive. It should
> call "gtk_container_forall", which does not occur even after the second
> (inner) call.
> Potentially you are just lacking some debug symbols here for this GTK
> library. note that "gtkwidget.c:0" is bogus already - the line numbers
> start with 1, so could we just skip those then? pass on all zero lined 
> inlines
> and not add/display them
> I guess so - but quite frankly I'm a bit uneasy to do this without 
> further
> exploration about the actual causes here. Afaik DWARF does not say 
> anything
> about the validity, so in theory there might be some language/compiler 
> that
> encodes valid DWARF locations with line 0?
> 
> Generally, it would be quite interesting to figure out why there is 
> /some/
> DWARF data here such that the code thinks it's able to decode the 
> inline frame
> but then fails and leads to this confusing result?
> right, but it's beyond my DWARF expertise ;-) we'll need
> to wait till one of you guys could take a look on it
> 
> thanks,
> jirka