Re: Wrong Perf Backtraces

From: ahmadkhorrami <ahmadkhorrami@ut.ac.ir>
To: Jiri Olsa <jolsa@redhat.com>
Cc: Milian Wolff <milian.wolff@kdab.com>,
	Steven Rostedt <rostedt@goodmis.org>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	Linux-trace Users <linux-trace-users@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	linux-trace-users-owner@vger.kernel.org,
	Jin Yao <yao.jin@linux.intel.com>,
	Namhyung Kim <namhyung@kernel.org>,
	Andi Kleen <ak@linux.intel.com>
Subject: Re: Wrong Perf Backtraces
Date: Tue, 31 Mar 2020 17:13:00 +0430
Message-ID: <cc4c1410059a672b81d27a1992cb5a9e@ut.ac.ir> (raw)
In-Reply-To: <8cb4f94b11fbac6ca3a44b924b6ee7b3@ut.ac.ir>

Hi,
Please confirm my thought in the previous email, if it is correct.
Regards.
On 2020-03-30 23:35, ahmadkhorrami wrote:

> Hi,
> Sorry, I did not pay attention to the fact that we need debug info to 
> be able to decode the callchain. So, without complete debug info the 
> backtrace (either the symbols or the addresses) is bogus. But you say 
> that the backtrace portions containing binaries with full debug info 
> are OK. Am I right? Please confirm if this is the case.
> If so, these portions may be enough for my current usecase.
> Regards.
> 
> On 2020-03-30 18:19, ahmadkhorrami wrote:
> 
> Hi,
> thanks, but I still do not understand the cause of the problem. It 
> seems that it is purely a software bug and the callchain addresses seem 
> wrong,by themselves. Am I right?
> Regards.
> On 2020-03-30 17:37, Jiri Olsa wrote:
> 
> On Mon, Mar 30, 2020 at 08:09:53AM +0200, Milian Wolff wrote: On 
> Sonntag, 29. März 2020 21:20:10 CEST Jiri Olsa wrote: On Sun, Mar 29, 
> 2020 at 03:50:57PM +0200, Milian Wolff wrote: On Sonntag, 29. März 2020 
> 14:39:33 CEST ahmadkhorrami wrote: Thanks. I did both of your changes. 
> Perhaps some outputs are revised.
> But I still have repeated function calls and the script detects them.
> Here is one of them when the sampling period is 1000 events: <snip>
> 
> Here, we have two consecutive "7f91788120b5
> gtk_widget_propagate_state+0x195
> (/usr/lib/x86_64-linux-gnu/libgtk-3.so.0.2200.30) gtkwidget.c:0" lines,
> while "gtk_widget_propagate_state+0x195" is not recursive. It should
> call "gtk_container_forall", which does not occur even after the second
> (inner) call.
> Potentially you are just lacking some debug symbols here for this GTK
> library. note that "gtkwidget.c:0" is bogus already - the line numbers
> start with 1, so could we just skip those then? pass on all zero lined 
> inlines
> and not add/display them
> I guess so - but quite frankly I'm a bit uneasy to do this without 
> further
> exploration about the actual causes here. Afaik DWARF does not say 
> anything
> about the validity, so in theory there might be some language/compiler 
> that
> encodes valid DWARF locations with line 0?
> 
> Generally, it would be quite interesting to figure out why there is 
> /some/
> DWARF data here such that the code thinks it's able to decode the 
> inline frame
> but then fails and leads to this confusing result?
> right, but it's beyond my DWARF expertise ;-) we'll need
> to wait till one of you guys could take a look on it
> 
> thanks,
> jirka