No source code for static library (compiled with -g -ggdb)

No source code for static library (compiled with -g -ggdb)

[Mark Davis]

Hello, I'm using perf record and perf report to profile an application
which is made from a static (.a) library file that I made as well as a
handful of C++ files that are not in the static library. They are all
compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
all linked together to create the application.

When I use perf report (in interactive/tui mode), when I annotate the
symbols that are in the C++ files (not the static library), I see the
source code intermingled with the assembly (this is what I want). I
can do this at any layer of the call stack. However, when I drill into
the symbols that are from the static library, I just see the assembly
of the leaf symbol. I don't expect to see the source in this case, as
the leaf symbol is a low-level library from libc; but, perf report
won't let me drill into (i.e., annotate) any non-leaf symbols.

Example:
execute_native_thread_routine
std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
Thread*)>::operator()()
main(int, char**)
my_func_A
my_func_B
malloc

When I annotate my_func_A and my_func_B, it just jumps to the
annotation of malloc (I see this listed at the top of the annotation
view). Note that my_func_A and my_func_B are both defined in the
static library that I'm linking in. However, when I do a similar thing
with a different stack where I'm annotating a function that's not from
the static library (but in the regular C++ files), it works fine.

Here's what I'm doing to create my static lib:
ar -cvrs $@ $(OBJS)


I did confirm with nm --debug-syms that my static lib has debugging
symbols in it:

U __assert_fail
0000000000000000 b .bss
0000000000000000 n .comment
U __cxa_atexit
0000000000000000 d .data
0000000000000000 N .debug_abbrev
0000000000000000 N .debug_aranges
0000000000000000 N .debug_info
0000000000000000 N .debug_line
0000000000000000 N .debug_loc
0000000000000000 N .debug_ranges
0000000000000000 N .debug_str
U __dso_handle
0000000000000000 r .eh_frame
U exit


How can I compile and link my application and configure perf report so
I can drill in like this on functions defined in a static library with
debug symbols?

Thank you,
Mark

Re: No source code for static library (compiled with -g -ggdb)

[Arnaldo Carvalho de Melo]

Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
> Hello, I'm using perf record and perf report to profile an application
> which is made from a static (.a) library file that I made as well as a
> handful of C++ files that are not in the static library. They are all
> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
> all linked together to create the application.
> 
> When I use perf report (in interactive/tui mode), when I annotate the
> symbols that are in the C++ files (not the static library), I see the
> source code intermingled with the assembly (this is what I want). I
> can do this at any layer of the call stack. However, when I drill into
> the symbols that are from the static library, I just see the assembly
> of the leaf symbol. I don't expect to see the source in this case, as
> the leaf symbol is a low-level library from libc; but, perf report
> won't let me drill into (i.e., annotate) any non-leaf symbols.
> 
> Example:
> execute_native_thread_routine
> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
> Thread*)>::operator()()
> main(int, char**)
> my_func_A
> my_func_B
> malloc

What happens when you press 'V'? Here, say for this callchain:

-    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
   - handlers_find
      - 0.68% signal_handlers_foreach_matched_R
           g_signal_handlers_disconnect_matched
           0x7f71fe809a80
           g_object_unref
         - st_widget_get_theme_node
              0.58% 0x7f71fe813429 

And I press V, I get:

-    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
   -→handlers_find libgobject-2.0.so.0.4600.2
      - 0.68% signal_handlers_foreach_matched_R libgobject-2.0.so.0.4600.2
           g_signal_handlers_disconnect_matched libgobject-2.0.so.0.4600.2
           0x7f71fe809a80 libgnome-shell.so
          →g_object_unref libgobject-2.0.so.0.4600.2
         -→st_widget_get_theme_node libgnome-shell.so
              0.58% 0x7f71fe813429 libgnome-shell.so 

See those arrows? They point to the places where you can annotate, the others
can't because they had no samples or were unresolved.

With some changes I think we can show the ones without samples, but that remains to be done.

Also, this is with:

[root@jouet ~]# perf --version
perf version 4.6.rc4.ga453697

This feature was introduced in:

commit 70e972788888315851a5c1b4982efbcafcd03b5b
Author: Arnaldo Carvalho de Melo <acme@redhat.com>
Date:   Thu Mar 19 16:07:21 2015 -0300

    perf hists browser: Indicate which callchain entries are annotated
    
    Now that we can annotate entries in a callchain, show which ones have an
    associated symbol and samples, by adding a right arrow just before the
    symbol name when in verbose mode.
    
    To toggle verbose mode press 'V'.

---------------------------------

Which is:

[acme@jouet linux]$ git tag --contains 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1
v4.1
[acme@jouet linux]$

Thanks,

- Arnaldo
 
> When I annotate my_func_A and my_func_B, it just jumps to the
> annotation of malloc (I see this listed at the top of the annotation
> view). Note that my_func_A and my_func_B are both defined in the
> static library that I'm linking in. However, when I do a similar thing
> with a different stack where I'm annotating a function that's not from
> the static library (but in the regular C++ files), it works fine.
> 
> Here's what I'm doing to create my static lib:
> ar -cvrs $@ $(OBJS)
> 
> 
> I did confirm with nm --debug-syms that my static lib has debugging
> symbols in it:
> 
> U __assert_fail
> 0000000000000000 b .bss
> 0000000000000000 n .comment
> U __cxa_atexit
> 0000000000000000 d .data
> 0000000000000000 N .debug_abbrev
> 0000000000000000 N .debug_aranges
> 0000000000000000 N .debug_info
> 0000000000000000 N .debug_line
> 0000000000000000 N .debug_loc
> 0000000000000000 N .debug_ranges
> 0000000000000000 N .debug_str
> U __dso_handle
> 0000000000000000 r .eh_frame
> U exit
> 
> 
> How can I compile and link my application and configure perf report so
> I can drill in like this on functions defined in a static library with
> debug symbols?
> 
> Thank you,
> Mark
> --
> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No source code for static library (compiled with -g -ggdb)

[Mark Davis]

Arnaldo, that's a really nice feature. Unfortunately, I'm currently
running on an older version of perf (3.13.11-ckt18) and it's a bit of
a pain to modify things like this on the cluster I'm running on
(although I can if I have something I absolutely need). So, for me
when I run verbose mode I simply get the DSO names in the callchains,
but no arrows. I would assume, if I had a newer version of perf, that
I would get no arrows for the symbols I want to annotate, given the
behavior I'm seeing.

Anything else I can try? Does anyone have any ideas about why this may
be happening? Again, this is only happening for symbols in my static
library, and I'm creating this library using:

ar -cvrs $@ $(OBJS)

(I'm not an expert on ar, so I could be doing this wrong?)

On Thu, Apr 28, 2016 at 6:16 PM, Arnaldo Carvalho de Melo
<arnaldo.melo@gmail.com> wrote:
> Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
>> Hello, I'm using perf record and perf report to profile an application
>> which is made from a static (.a) library file that I made as well as a
>> handful of C++ files that are not in the static library. They are all
>> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
>> all linked together to create the application.
>>
>> When I use perf report (in interactive/tui mode), when I annotate the
>> symbols that are in the C++ files (not the static library), I see the
>> source code intermingled with the assembly (this is what I want). I
>> can do this at any layer of the call stack. However, when I drill into
>> the symbols that are from the static library, I just see the assembly
>> of the leaf symbol. I don't expect to see the source in this case, as
>> the leaf symbol is a low-level library from libc; but, perf report
>> won't let me drill into (i.e., annotate) any non-leaf symbols.
>>
>> Example:
>> execute_native_thread_routine
>> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
>> Thread*)>::operator()()
>> main(int, char**)
>> my_func_A
>> my_func_B
>> malloc
>
> What happens when you press 'V'? Here, say for this callchain:
>
> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
>    - handlers_find
>       - 0.68% signal_handlers_foreach_matched_R
>            g_signal_handlers_disconnect_matched
>            0x7f71fe809a80
>            g_object_unref
>          - st_widget_get_theme_node
>               0.58% 0x7f71fe813429
>
> And I press V, I get:
>
> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
>    -→handlers_find libgobject-2.0.so.0.4600.2
>       - 0.68% signal_handlers_foreach_matched_R libgobject-2.0.so.0.4600.2
>            g_signal_handlers_disconnect_matched libgobject-2.0.so.0.4600.2
>            0x7f71fe809a80 libgnome-shell.so
>           →g_object_unref libgobject-2.0.so.0.4600.2
>          -→st_widget_get_theme_node libgnome-shell.so
>               0.58% 0x7f71fe813429 libgnome-shell.so
>
> See those arrows? They point to the places where you can annotate, the others
> can't because they had no samples or were unresolved.
>
> With some changes I think we can show the ones without samples, but that remains to be done.
>
> Also, this is with:
>
> [root@jouet ~]# perf --version
> perf version 4.6.rc4.ga453697
>
> This feature was introduced in:
>
> commit 70e972788888315851a5c1b4982efbcafcd03b5b
> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> Date:   Thu Mar 19 16:07:21 2015 -0300
>
>     perf hists browser: Indicate which callchain entries are annotated
>
>     Now that we can annotate entries in a callchain, show which ones have an
>     associated symbol and samples, by adding a right arrow just before the
>     symbol name when in verbose mode.
>
>     To toggle verbose mode press 'V'.
>
> ---------------------------------
>
> Which is:
>
> [acme@jouet linux]$ git tag --contains 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1
> v4.1
> [acme@jouet linux]$
>
> Thanks,
>
> - Arnaldo
>
>> When I annotate my_func_A and my_func_B, it just jumps to the
>> annotation of malloc (I see this listed at the top of the annotation
>> view). Note that my_func_A and my_func_B are both defined in the
>> static library that I'm linking in. However, when I do a similar thing
>> with a different stack where I'm annotating a function that's not from
>> the static library (but in the regular C++ files), it works fine.
>>
>> Here's what I'm doing to create my static lib:
>> ar -cvrs $@ $(OBJS)
>>
>>
>> I did confirm with nm --debug-syms that my static lib has debugging
>> symbols in it:
>>
>> U __assert_fail
>> 0000000000000000 b .bss
>> 0000000000000000 n .comment
>> U __cxa_atexit
>> 0000000000000000 d .data
>> 0000000000000000 N .debug_abbrev
>> 0000000000000000 N .debug_aranges
>> 0000000000000000 N .debug_info
>> 0000000000000000 N .debug_line
>> 0000000000000000 N .debug_loc
>> 0000000000000000 N .debug_ranges
>> 0000000000000000 N .debug_str
>> U __dso_handle
>> 0000000000000000 r .eh_frame
>> U exit
>>
>>
>> How can I compile and link my application and configure perf report so
>> I can drill in like this on functions defined in a static library with
>> debug symbols?
>>
>> Thank you,
>> Mark
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No source code for static library (compiled with -g -ggdb)

[Mark Davis]

It turns out that I was incorrect about diagnosing this problem, and
this is probably just due to my misunderstanding of how to use
annotations in perf. (Sorry, I'm a new user to perf and trying to get
my head around it.) I do have a question below.

Recall my example from above:

execute_native_thread_routine
std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
Thread*)>::operator()()
main(int, char**)
my_func_A
my_func_B
malloc

I was trying to jump into my_func_B to see the annotation of it, and
it kept jumping me into the annotation of malloc. That is, I wanted to
see *where* malloc was being called from my_func_B, and the hottest
areas of my_func_B. But, when I type "a" when on my_func_B (or drill
in with the arrows or enter key), it always takes me to the annotation
of malloc (i.e., the leaf node in the callchain). I figured this out
by filtering to the DSO of my system (which made the leaf node always
be a symbol in my system as opposed to some system library, etc.) and
it shows that my symbols have debug info, etc.

But if I wanted to see the annotation of my_func_B, how would I? Is it
possible that I'm not able to see this simply because I don't have
enough samples of it (and all the real time is spent down in malloc)?
Or is there some other reason? Arnaldo, what exactly determines if
your arrows show up when pressing V in perf v4.1?

Furthermore, is there some other feature of perf_events that I should
try (perhaps tracing?) to get more fidelity? My programs are
relatively short-running, so I can afford to wait a bit longer.

Thank you for any input you can provide as I'm learning perf_events.

Mark



On Fri, Apr 29, 2016 at 8:28 AM, Mark Davis <markdavisinboston@gmail.com> wrote:
> Arnaldo, that's a really nice feature. Unfortunately, I'm currently
> running on an older version of perf (3.13.11-ckt18) and it's a bit of
> a pain to modify things like this on the cluster I'm running on
> (although I can if I have something I absolutely need). So, for me
> when I run verbose mode I simply get the DSO names in the callchains,
> but no arrows. I would assume, if I had a newer version of perf, that
> I would get no arrows for the symbols I want to annotate, given the
> behavior I'm seeing.
>
> Anything else I can try? Does anyone have any ideas about why this may
> be happening? Again, this is only happening for symbols in my static
> library, and I'm creating this library using:
>
> ar -cvrs $@ $(OBJS)
>
> (I'm not an expert on ar, so I could be doing this wrong?)
>
> On Thu, Apr 28, 2016 at 6:16 PM, Arnaldo Carvalho de Melo
> <arnaldo.melo@gmail.com> wrote:
>> Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
>>> Hello, I'm using perf record and perf report to profile an application
>>> which is made from a static (.a) library file that I made as well as a
>>> handful of C++ files that are not in the static library. They are all
>>> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
>>> all linked together to create the application.
>>>
>>> When I use perf report (in interactive/tui mode), when I annotate the
>>> symbols that are in the C++ files (not the static library), I see the
>>> source code intermingled with the assembly (this is what I want). I
>>> can do this at any layer of the call stack. However, when I drill into
>>> the symbols that are from the static library, I just see the assembly
>>> of the leaf symbol. I don't expect to see the source in this case, as
>>> the leaf symbol is a low-level library from libc; but, perf report
>>> won't let me drill into (i.e., annotate) any non-leaf symbols.
>>>
>>> Example:
>>> execute_native_thread_routine
>>> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
>>> Thread*)>::operator()()
>>> main(int, char**)
>>> my_func_A
>>> my_func_B
>>> malloc
>>
>> What happens when you press 'V'? Here, say for this callchain:
>>
>> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
>>    - handlers_find
>>       - 0.68% signal_handlers_foreach_matched_R
>>            g_signal_handlers_disconnect_matched
>>            0x7f71fe809a80
>>            g_object_unref
>>          - st_widget_get_theme_node
>>               0.58% 0x7f71fe813429
>>
>> And I press V, I get:
>>
>> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
>>    -→handlers_find libgobject-2.0.so.0.4600.2
>>       - 0.68% signal_handlers_foreach_matched_R libgobject-2.0.so.0.4600.2
>>            g_signal_handlers_disconnect_matched libgobject-2.0.so.0.4600.2
>>            0x7f71fe809a80 libgnome-shell.so
>>           →g_object_unref libgobject-2.0.so.0.4600.2
>>          -→st_widget_get_theme_node libgnome-shell.so
>>               0.58% 0x7f71fe813429 libgnome-shell.so
>>
>> See those arrows? They point to the places where you can annotate, the others
>> can't because they had no samples or were unresolved.
>>
>> With some changes I think we can show the ones without samples, but that remains to be done.
>>
>> Also, this is with:
>>
>> [root@jouet ~]# perf --version
>> perf version 4.6.rc4.ga453697
>>
>> This feature was introduced in:
>>
>> commit 70e972788888315851a5c1b4982efbcafcd03b5b
>> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>> Date:   Thu Mar 19 16:07:21 2015 -0300
>>
>>     perf hists browser: Indicate which callchain entries are annotated
>>
>>     Now that we can annotate entries in a callchain, show which ones have an
>>     associated symbol and samples, by adding a right arrow just before the
>>     symbol name when in verbose mode.
>>
>>     To toggle verbose mode press 'V'.
>>
>> ---------------------------------
>>
>> Which is:
>>
>> [acme@jouet linux]$ git tag --contains 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1
>> v4.1
>> [acme@jouet linux]$
>>
>> Thanks,
>>
>> - Arnaldo
>>
>>> When I annotate my_func_A and my_func_B, it just jumps to the
>>> annotation of malloc (I see this listed at the top of the annotation
>>> view). Note that my_func_A and my_func_B are both defined in the
>>> static library that I'm linking in. However, when I do a similar thing
>>> with a different stack where I'm annotating a function that's not from
>>> the static library (but in the regular C++ files), it works fine.
>>>
>>> Here's what I'm doing to create my static lib:
>>> ar -cvrs $@ $(OBJS)
>>>
>>>
>>> I did confirm with nm --debug-syms that my static lib has debugging
>>> symbols in it:
>>>
>>> U __assert_fail
>>> 0000000000000000 b .bss
>>> 0000000000000000 n .comment
>>> U __cxa_atexit
>>> 0000000000000000 d .data
>>> 0000000000000000 N .debug_abbrev
>>> 0000000000000000 N .debug_aranges
>>> 0000000000000000 N .debug_info
>>> 0000000000000000 N .debug_line
>>> 0000000000000000 N .debug_loc
>>> 0000000000000000 N .debug_ranges
>>> 0000000000000000 N .debug_str
>>> U __dso_handle
>>> 0000000000000000 r .eh_frame
>>> U exit
>>>
>>>
>>> How can I compile and link my application and configure perf report so
>>> I can drill in like this on functions defined in a static library with
>>> debug symbols?
>>>
>>> Thank you,
>>> Mark
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No source code for static library (compiled with -g -ggdb)

[Arnaldo Carvalho de Melo]

Em Fri, Apr 29, 2016 at 08:28:29AM -0400, Mark Davis escreveu:
> Arnaldo, that's a really nice feature. Unfortunately, I'm currently
> running on an older version of perf (3.13.11-ckt18) and it's a bit of
> a pain to modify things like this on the cluster I'm running on
> (although I can if I have something I absolutely need). So, for me

Well, its not _that_ difficult to use a newer perf tool on whatever
kernel you are using, and that should work, as the new tool should
notice whatever features may be missing for something you ask it to do.

So, you can do:

[acme@jouet linux]$ make help | grep perf
  perf-tar-src-pkg    - Build perf-4.6.0-rc4.tar source tarball
  perf-targz-src-pkg  - Build perf-4.6.0-rc4.tar.gz source tarball
  perf-tarbz2-src-pkg - Build perf-4.6.0-rc4.tar.bz2 source tarball
  perf-tarxz-src-pkg  - Build perf-4.6.0-rc4.tar.xz source tarball
[acme@jouet linux]$ 

Do any of this, say:

  make perf-targz-src-pkg

On a recent kernel source, and you'll get a tarball with what you need
to build it on the target system.

Then just use it as ./perf or make sure that the binary is in your path.

> when I run verbose mode I simply get the DSO names in the callchains,
> but no arrows. I would assume, if I had a newer version of perf, that
> I would get no arrows for the symbols I want to annotate, given the
> behavior I'm seeing.

Yes, this seems to be the case.
 
> Anything else I can try? Does anyone have any ideas about why this may
> be happening? Again, this is only happening for symbols in my static
> library, and I'm creating this library using:

I'll read the other message you sent, I think you clarified things
further there...
 
> ar -cvrs $@ $(OBJS)
> 
> (I'm not an expert on ar, so I could be doing this wrong?)
> 
> On Thu, Apr 28, 2016 at 6:16 PM, Arnaldo Carvalho de Melo
> <arnaldo.melo@gmail.com> wrote:
> > Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
> >> Hello, I'm using perf record and perf report to profile an application
> >> which is made from a static (.a) library file that I made as well as a
> >> handful of C++ files that are not in the static library. They are all
> >> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
> >> all linked together to create the application.
> >>
> >> When I use perf report (in interactive/tui mode), when I annotate the
> >> symbols that are in the C++ files (not the static library), I see the
> >> source code intermingled with the assembly (this is what I want). I
> >> can do this at any layer of the call stack. However, when I drill into
> >> the symbols that are from the static library, I just see the assembly
> >> of the leaf symbol. I don't expect to see the source in this case, as
> >> the leaf symbol is a low-level library from libc; but, perf report
> >> won't let me drill into (i.e., annotate) any non-leaf symbols.
> >>
> >> Example:
> >> execute_native_thread_routine
> >> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
> >> Thread*)>::operator()()
> >> main(int, char**)
> >> my_func_A
> >> my_func_B
> >> malloc
> >
> > What happens when you press 'V'? Here, say for this callchain:
> >
> > -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
> >    - handlers_find
> >       - 0.68% signal_handlers_foreach_matched_R
> >            g_signal_handlers_disconnect_matched
> >            0x7f71fe809a80
> >            g_object_unref
> >          - st_widget_get_theme_node
> >               0.58% 0x7f71fe813429
> >
> > And I press V, I get:
> >
> > -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
> >    -→handlers_find libgobject-2.0.so.0.4600.2
> >       - 0.68% signal_handlers_foreach_matched_R libgobject-2.0.so.0.4600.2
> >            g_signal_handlers_disconnect_matched libgobject-2.0.so.0.4600.2
> >            0x7f71fe809a80 libgnome-shell.so
> >           →g_object_unref libgobject-2.0.so.0.4600.2
> >          -→st_widget_get_theme_node libgnome-shell.so
> >               0.58% 0x7f71fe813429 libgnome-shell.so
> >
> > See those arrows? They point to the places where you can annotate, the others
> > can't because they had no samples or were unresolved.
> >
> > With some changes I think we can show the ones without samples, but that remains to be done.
> >
> > Also, this is with:
> >
> > [root@jouet ~]# perf --version
> > perf version 4.6.rc4.ga453697
> >
> > This feature was introduced in:
> >
> > commit 70e972788888315851a5c1b4982efbcafcd03b5b
> > Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> > Date:   Thu Mar 19 16:07:21 2015 -0300
> >
> >     perf hists browser: Indicate which callchain entries are annotated
> >
> >     Now that we can annotate entries in a callchain, show which ones have an
> >     associated symbol and samples, by adding a right arrow just before the
> >     symbol name when in verbose mode.
> >
> >     To toggle verbose mode press 'V'.
> >
> > ---------------------------------
> >
> > Which is:
> >
> > [acme@jouet linux]$ git tag --contains 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1
> > v4.1
> > [acme@jouet linux]$
> >
> > Thanks,
> >
> > - Arnaldo
> >
> >> When I annotate my_func_A and my_func_B, it just jumps to the
> >> annotation of malloc (I see this listed at the top of the annotation
> >> view). Note that my_func_A and my_func_B are both defined in the
> >> static library that I'm linking in. However, when I do a similar thing
> >> with a different stack where I'm annotating a function that's not from
> >> the static library (but in the regular C++ files), it works fine.
> >>
> >> Here's what I'm doing to create my static lib:
> >> ar -cvrs $@ $(OBJS)
> >>
> >>
> >> I did confirm with nm --debug-syms that my static lib has debugging
> >> symbols in it:
> >>
> >> U __assert_fail
> >> 0000000000000000 b .bss
> >> 0000000000000000 n .comment
> >> U __cxa_atexit
> >> 0000000000000000 d .data
> >> 0000000000000000 N .debug_abbrev
> >> 0000000000000000 N .debug_aranges
> >> 0000000000000000 N .debug_info
> >> 0000000000000000 N .debug_line
> >> 0000000000000000 N .debug_loc
> >> 0000000000000000 N .debug_ranges
> >> 0000000000000000 N .debug_str
> >> U __dso_handle
> >> 0000000000000000 r .eh_frame
> >> U exit
> >>
> >>
> >> How can I compile and link my application and configure perf report so
> >> I can drill in like this on functions defined in a static library with
> >> debug symbols?
> >>
> >> Thank you,
> >> Mark
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
> >> the body of a message to majordomo@vger.kernel.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No source code for static library (compiled with -g -ggdb)

[Mark Davis]

Arnaldo, I was able to install perf version 4.6.rc5.g1b46bac and am
able to see the arrows to indicate the annotated symbols. That is a
very useful feature. Furthermore, I'm now able to see exactly what I
wanted to see before -- annotated source of non-leaf symbols in the
callchain. So, as of now I think I'm all set.

Thank you,
Mark

On Fri, Apr 29, 2016 at 10:45 AM, Arnaldo Carvalho de Melo
<arnaldo.melo@gmail.com> wrote:
> Em Fri, Apr 29, 2016 at 08:28:29AM -0400, Mark Davis escreveu:
>> Arnaldo, that's a really nice feature. Unfortunately, I'm currently
>> running on an older version of perf (3.13.11-ckt18) and it's a bit of
>> a pain to modify things like this on the cluster I'm running on
>> (although I can if I have something I absolutely need). So, for me
>
> Well, its not _that_ difficult to use a newer perf tool on whatever
> kernel you are using, and that should work, as the new tool should
> notice whatever features may be missing for something you ask it to do.
>
> So, you can do:
>
> [acme@jouet linux]$ make help | grep perf
>   perf-tar-src-pkg    - Build perf-4.6.0-rc4.tar source tarball
>   perf-targz-src-pkg  - Build perf-4.6.0-rc4.tar.gz source tarball
>   perf-tarbz2-src-pkg - Build perf-4.6.0-rc4.tar.bz2 source tarball
>   perf-tarxz-src-pkg  - Build perf-4.6.0-rc4.tar.xz source tarball
> [acme@jouet linux]$
>
> Do any of this, say:
>
>   make perf-targz-src-pkg
>
> On a recent kernel source, and you'll get a tarball with what you need
> to build it on the target system.
>
> Then just use it as ./perf or make sure that the binary is in your path.
>
>> when I run verbose mode I simply get the DSO names in the callchains,
>> but no arrows. I would assume, if I had a newer version of perf, that
>> I would get no arrows for the symbols I want to annotate, given the
>> behavior I'm seeing.
>
> Yes, this seems to be the case.
>
>> Anything else I can try? Does anyone have any ideas about why this may
>> be happening? Again, this is only happening for symbols in my static
>> library, and I'm creating this library using:
>
> I'll read the other message you sent, I think you clarified things
> further there...
>
>> ar -cvrs $@ $(OBJS)
>>
>> (I'm not an expert on ar, so I could be doing this wrong?)
>>
>> On Thu, Apr 28, 2016 at 6:16 PM, Arnaldo Carvalho de Melo
>> <arnaldo.melo@gmail.com> wrote:
>> > Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
>> >> Hello, I'm using perf record and perf report to profile an application
>> >> which is made from a static (.a) library file that I made as well as a
>> >> handful of C++ files that are not in the static library. They are all
>> >> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
>> >> all linked together to create the application.
>> >>
>> >> When I use perf report (in interactive/tui mode), when I annotate the
>> >> symbols that are in the C++ files (not the static library), I see the
>> >> source code intermingled with the assembly (this is what I want). I
>> >> can do this at any layer of the call stack. However, when I drill into
>> >> the symbols that are from the static library, I just see the assembly
>> >> of the leaf symbol. I don't expect to see the source in this case, as
>> >> the leaf symbol is a low-level library from libc; but, perf report
>> >> won't let me drill into (i.e., annotate) any non-leaf symbols.
>> >>
>> >> Example:
>> >> execute_native_thread_routine
>> >> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
>> >> Thread*)>::operator()()
>> >> main(int, char**)
>> >> my_func_A
>> >> my_func_B
>> >> malloc
>> >
>> > What happens when you press 'V'? Here, say for this callchain:
>> >
>> > -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
>> >    - handlers_find
>> >       - 0.68% signal_handlers_foreach_matched_R
>> >            g_signal_handlers_disconnect_matched
>> >            0x7f71fe809a80
>> >            g_object_unref
>> >          - st_widget_get_theme_node
>> >               0.58% 0x7f71fe813429
>> >
>> > And I press V, I get:
>> >
>> > -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.] handlers_find
>> >    -→handlers_find libgobject-2.0.so.0.4600.2
>> >       - 0.68% signal_handlers_foreach_matched_R libgobject-2.0.so.0.4600.2
>> >            g_signal_handlers_disconnect_matched libgobject-2.0.so.0.4600.2
>> >            0x7f71fe809a80 libgnome-shell.so
>> >           →g_object_unref libgobject-2.0.so.0.4600.2
>> >          -→st_widget_get_theme_node libgnome-shell.so
>> >               0.58% 0x7f71fe813429 libgnome-shell.so
>> >
>> > See those arrows? They point to the places where you can annotate, the others
>> > can't because they had no samples or were unresolved.
>> >
>> > With some changes I think we can show the ones without samples, but that remains to be done.
>> >
>> > Also, this is with:
>> >
>> > [root@jouet ~]# perf --version
>> > perf version 4.6.rc4.ga453697
>> >
>> > This feature was introduced in:
>> >
>> > commit 70e972788888315851a5c1b4982efbcafcd03b5b
>> > Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>> > Date:   Thu Mar 19 16:07:21 2015 -0300
>> >
>> >     perf hists browser: Indicate which callchain entries are annotated
>> >
>> >     Now that we can annotate entries in a callchain, show which ones have an
>> >     associated symbol and samples, by adding a right arrow just before the
>> >     symbol name when in verbose mode.
>> >
>> >     To toggle verbose mode press 'V'.
>> >
>> > ---------------------------------
>> >
>> > Which is:
>> >
>> > [acme@jouet linux]$ git tag --contains 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1
>> > v4.1
>> > [acme@jouet linux]$
>> >
>> > Thanks,
>> >
>> > - Arnaldo
>> >
>> >> When I annotate my_func_A and my_func_B, it just jumps to the
>> >> annotation of malloc (I see this listed at the top of the annotation
>> >> view). Note that my_func_A and my_func_B are both defined in the
>> >> static library that I'm linking in. However, when I do a similar thing
>> >> with a different stack where I'm annotating a function that's not from
>> >> the static library (but in the regular C++ files), it works fine.
>> >>
>> >> Here's what I'm doing to create my static lib:
>> >> ar -cvrs $@ $(OBJS)
>> >>
>> >>
>> >> I did confirm with nm --debug-syms that my static lib has debugging
>> >> symbols in it:
>> >>
>> >> U __assert_fail
>> >> 0000000000000000 b .bss
>> >> 0000000000000000 n .comment
>> >> U __cxa_atexit
>> >> 0000000000000000 d .data
>> >> 0000000000000000 N .debug_abbrev
>> >> 0000000000000000 N .debug_aranges
>> >> 0000000000000000 N .debug_info
>> >> 0000000000000000 N .debug_line
>> >> 0000000000000000 N .debug_loc
>> >> 0000000000000000 N .debug_ranges
>> >> 0000000000000000 N .debug_str
>> >> U __dso_handle
>> >> 0000000000000000 r .eh_frame
>> >> U exit
>> >>
>> >>
>> >> How can I compile and link my application and configure perf report so
>> >> I can drill in like this on functions defined in a static library with
>> >> debug symbols?
>> >>
>> >> Thank you,
>> >> Mark
>> >> --
>> >> To unsubscribe from this list: send the line "unsubscribe linux-perf-users" in
>> >> the body of a message to majordomo@vger.kernel.org
>> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: No source code for static library (compiled with -g -ggdb)

[Milian Wolff]

[-- Attachment #1: Type: text/plain, Size: 9790 bytes --]

On Freitag, 29. April 2016 09:26:44 CEST Mark Davis wrote:
> It turns out that I was incorrect about diagnosing this problem, and
> this is probably just due to my misunderstanding of how to use
> annotations in perf. (Sorry, I'm a new user to perf and trying to get
> my head around it.) I do have a question below.
> 
> Recall my example from above:
> 
> execute_native_thread_routine
> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
> Thread*)>::operator()()
> main(int, char**)
> my_func_A
> my_func_B
> malloc
> 
> I was trying to jump into my_func_B to see the annotation of it, and
> it kept jumping me into the annotation of malloc. That is, I wanted to
> see *where* malloc was being called from my_func_B, and the hottest
> areas of my_func_B. But, when I type "a" when on my_func_B (or drill
> in with the arrows or enter key), it always takes me to the annotation
> of malloc (i.e., the leaf node in the callchain). I figured this out
> by filtering to the DSO of my system (which made the leaf node always
> be a symbol in my system as opposed to some system library, etc.) and
> it shows that my symbols have debug info, etc.
> 
> But if I wanted to see the annotation of my_func_B, how would I? Is it
> possible that I'm not able to see this simply because I don't have
> enough samples of it (and all the real time is spent down in malloc)?
> Or is there some other reason? Arnaldo, what exactly determines if
> your arrows show up when pressing V in perf v4.1?

I'm not really sure, but I seem to remember this "bug" in older perf versions. 
I see in the other thread that you manually installed a newer perf, but could 
you check whether the following works with the older versions:

- navigate to the function you want to annotate (my_func_B)
- instead of pressing 'a', press 'arrow right'
- in the menu that shows up, select "annotate"

I /think/ that 'a' used to be the "annotate sample entry point" hotkey as you 
encounter, whereas the submenu always allowed one to annotate individual 
frames of the sample callstack.

> Furthermore, is there some other feature of perf_events that I should
> try (perhaps tracing?) to get more fidelity? My programs are
> relatively short-running, so I can afford to wait a bit longer.

Tracing will require you to add tracepoints via uprobe manually, which is 
super time consuming - the changeset for Systemtap trace point support in perf 
has not yet been merged (bummer!).

I suggest you try to increase the sampling rate if you are interested in more 
details of short-running apps.

> Thank you for any input you can provide as I'm learning perf_events.

On more suggestion from my side: In your original mail you said you compile 
your code with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer". I think 
this will produce potentially extremely misleading results, due to you 
disabling the arguably most useful optimization feature of a compiler there 
is: "-fno-inline"

I /really/ urge you to remove this check and actually profile a "real" world 
example that is build with "-g -O2", i.e. with inlining enabled. Yes, the 
callstacks /may/ be harder to grasp, but at the same time you actually profile 
what the real app is doing. In C++ code, esp. with templates, I've seen 
hotspots disappear just due to inlining and the avalanche effect they can 
trigger in a good optimizer, e.g. by enabling deadcode elimination etc. 

Cheers

> On Fri, Apr 29, 2016 at 8:28 AM, Mark Davis <markdavisinboston@gmail.com> 
wrote:
> > Arnaldo, that's a really nice feature. Unfortunately, I'm currently
> > running on an older version of perf (3.13.11-ckt18) and it's a bit of
> > a pain to modify things like this on the cluster I'm running on
> > (although I can if I have something I absolutely need). So, for me
> > when I run verbose mode I simply get the DSO names in the callchains,
> > but no arrows. I would assume, if I had a newer version of perf, that
> > I would get no arrows for the symbols I want to annotate, given the
> > behavior I'm seeing.
> > 
> > Anything else I can try? Does anyone have any ideas about why this may
> > be happening? Again, this is only happening for symbols in my static
> > library, and I'm creating this library using:
> > 
> > ar -cvrs $@ $(OBJS)
> > 
> > (I'm not an expert on ar, so I could be doing this wrong?)
> > 
> > On Thu, Apr 28, 2016 at 6:16 PM, Arnaldo Carvalho de Melo
> > 
> > <arnaldo.melo@gmail.com> wrote:
> >> Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
> >>> Hello, I'm using perf record and perf report to profile an application
> >>> which is made from a static (.a) library file that I made as well as a
> >>> handful of C++ files that are not in the static library. They are all
> >>> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
> >>> all linked together to create the application.
> >>> 
> >>> When I use perf report (in interactive/tui mode), when I annotate the
> >>> symbols that are in the C++ files (not the static library), I see the
> >>> source code intermingled with the assembly (this is what I want). I
> >>> can do this at any layer of the call stack. However, when I drill into
> >>> the symbols that are from the static library, I just see the assembly
> >>> of the leaf symbol. I don't expect to see the source in this case, as
> >>> the leaf symbol is a low-level library from libc; but, perf report
> >>> won't let me drill into (i.e., annotate) any non-leaf symbols.
> >>> 
> >>> Example:
> >>> execute_native_thread_routine
> >>> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
> >>> Thread*)>::operator()()
> >>> main(int, char**)
> >>> my_func_A
> >>> my_func_B
> >>> malloc
> >> 
> >> What happens when you press 'V'? Here, say for this callchain:
> >> 
> >> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.]
> >> handlers_find>> 
> >>    - handlers_find
> >>    
> >>       - 0.68% signal_handlers_foreach_matched_R
> >>       
> >>            g_signal_handlers_disconnect_matched
> >>            0x7f71fe809a80
> >>            g_object_unref
> >>          
> >>          - st_widget_get_theme_node
> >>          
> >>               0.58% 0x7f71fe813429
> >> 
> >> And I press V, I get:
> >> 
> >> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.]
> >> handlers_find>> 
> >>    -→handlers_find libgobject-2.0.so.0.4600.2
> >>    
> >>       - 0.68% signal_handlers_foreach_matched_R
> >>       libgobject-2.0.so.0.4600.2
> >>       
> >>            g_signal_handlers_disconnect_matched
> >>            libgobject-2.0.so.0.4600.2
> >>            0x7f71fe809a80 libgnome-shell.so
> >>           
> >>           →g_object_unref libgobject-2.0.so.0.4600.2
> >>          
> >>          -→st_widget_get_theme_node libgnome-shell.so
> >>          
> >>               0.58% 0x7f71fe813429 libgnome-shell.so
> >> 
> >> See those arrows? They point to the places where you can annotate, the
> >> others can't because they had no samples or were unresolved.
> >> 
> >> With some changes I think we can show the ones without samples, but that
> >> remains to be done.
> >> 
> >> Also, this is with:
> >> 
> >> [root@jouet ~]# perf --version
> >> perf version 4.6.rc4.ga453697
> >> 
> >> This feature was introduced in:
> >> 
> >> commit 70e972788888315851a5c1b4982efbcafcd03b5b
> >> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
> >> Date:   Thu Mar 19 16:07:21 2015 -0300
> >> 
> >>     perf hists browser: Indicate which callchain entries are annotated
> >>     
> >>     Now that we can annotate entries in a callchain, show which ones have
> >>     an
> >>     associated symbol and samples, by adding a right arrow just before
> >>     the
> >>     symbol name when in verbose mode.
> >>     
> >>     To toggle verbose mode press 'V'.
> >> 
> >> ---------------------------------
> >> 
> >> Which is:
> >> 
> >> [acme@jouet linux]$ git tag --contains
> >> 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1 v4.1
> >> [acme@jouet linux]$
> >> 
> >> Thanks,
> >> 
> >> - Arnaldo
> >> 
> >>> When I annotate my_func_A and my_func_B, it just jumps to the
> >>> annotation of malloc (I see this listed at the top of the annotation
> >>> view). Note that my_func_A and my_func_B are both defined in the
> >>> static library that I'm linking in. However, when I do a similar thing
> >>> with a different stack where I'm annotating a function that's not from
> >>> the static library (but in the regular C++ files), it works fine.
> >>> 
> >>> Here's what I'm doing to create my static lib:
> >>> ar -cvrs $@ $(OBJS)
> >>> 
> >>> 
> >>> I did confirm with nm --debug-syms that my static lib has debugging
> >>> symbols in it:
> >>> 
> >>> U __assert_fail
> >>> 0000000000000000 b .bss
> >>> 0000000000000000 n .comment
> >>> U __cxa_atexit
> >>> 0000000000000000 d .data
> >>> 0000000000000000 N .debug_abbrev
> >>> 0000000000000000 N .debug_aranges
> >>> 0000000000000000 N .debug_info
> >>> 0000000000000000 N .debug_line
> >>> 0000000000000000 N .debug_loc
> >>> 0000000000000000 N .debug_ranges
> >>> 0000000000000000 N .debug_str
> >>> U __dso_handle
> >>> 0000000000000000 r .eh_frame
> >>> U exit
> >>> 
> >>> 
> >>> How can I compile and link my application and configure perf report so
> >>> I can drill in like this on functions defined in a static library with
> >>> debug symbols?
> >>> 
> >>> Thank you,
> >>> Mark



-- 
Milian Wolff | milian.wolff@kdab.com | Software Engineer
KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
Tel: +49-30-521325470
KDAB - The Qt Experts

[-- Attachment #2: smime.p7s --]
[-- Type: application/pkcs7-signature, Size: 5903 bytes --]

Re: No source code for static library (compiled with -g -ggdb)

[Mark Davis]

Milian, this is a very slow reply -- but I did want to thank you for
your feedback. I did remove the -fno-inline flag when profiling and it
does give more useful results.

On Mon, May 2, 2016 at 3:58 AM, Milian Wolff <milian.wolff@kdab.com> wrote:
> On Freitag, 29. April 2016 09:26:44 CEST Mark Davis wrote:
>> It turns out that I was incorrect about diagnosing this problem, and
>> this is probably just due to my misunderstanding of how to use
>> annotations in perf. (Sorry, I'm a new user to perf and trying to get
>> my head around it.) I do have a question below.
>>
>> Recall my example from above:
>>
>> execute_native_thread_routine
>> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
>> Thread*)>::operator()()
>> main(int, char**)
>> my_func_A
>> my_func_B
>> malloc
>>
>> I was trying to jump into my_func_B to see the annotation of it, and
>> it kept jumping me into the annotation of malloc. That is, I wanted to
>> see *where* malloc was being called from my_func_B, and the hottest
>> areas of my_func_B. But, when I type "a" when on my_func_B (or drill
>> in with the arrows or enter key), it always takes me to the annotation
>> of malloc (i.e., the leaf node in the callchain). I figured this out
>> by filtering to the DSO of my system (which made the leaf node always
>> be a symbol in my system as opposed to some system library, etc.) and
>> it shows that my symbols have debug info, etc.
>>
>> But if I wanted to see the annotation of my_func_B, how would I? Is it
>> possible that I'm not able to see this simply because I don't have
>> enough samples of it (and all the real time is spent down in malloc)?
>> Or is there some other reason? Arnaldo, what exactly determines if
>> your arrows show up when pressing V in perf v4.1?
>
> I'm not really sure, but I seem to remember this "bug" in older perf versions.
> I see in the other thread that you manually installed a newer perf, but could
> you check whether the following works with the older versions:
>
> - navigate to the function you want to annotate (my_func_B)
> - instead of pressing 'a', press 'arrow right'
> - in the menu that shows up, select "annotate"
>
> I /think/ that 'a' used to be the "annotate sample entry point" hotkey as you
> encounter, whereas the submenu always allowed one to annotate individual
> frames of the sample callstack.
>
>> Furthermore, is there some other feature of perf_events that I should
>> try (perhaps tracing?) to get more fidelity? My programs are
>> relatively short-running, so I can afford to wait a bit longer.
>
> Tracing will require you to add tracepoints via uprobe manually, which is
> super time consuming - the changeset for Systemtap trace point support in perf
> has not yet been merged (bummer!).
>
> I suggest you try to increase the sampling rate if you are interested in more
> details of short-running apps.
>
>> Thank you for any input you can provide as I'm learning perf_events.
>
> On more suggestion from my side: In your original mail you said you compile
> your code with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer". I think
> this will produce potentially extremely misleading results, due to you
> disabling the arguably most useful optimization feature of a compiler there
> is: "-fno-inline"
>
> I /really/ urge you to remove this check and actually profile a "real" world
> example that is build with "-g -O2", i.e. with inlining enabled. Yes, the
> callstacks /may/ be harder to grasp, but at the same time you actually profile
> what the real app is doing. In C++ code, esp. with templates, I've seen
> hotspots disappear just due to inlining and the avalanche effect they can
> trigger in a good optimizer, e.g. by enabling deadcode elimination etc.
>
> Cheers
>
>> On Fri, Apr 29, 2016 at 8:28 AM, Mark Davis <markdavisinboston@gmail.com>
> wrote:
>> > Arnaldo, that's a really nice feature. Unfortunately, I'm currently
>> > running on an older version of perf (3.13.11-ckt18) and it's a bit of
>> > a pain to modify things like this on the cluster I'm running on
>> > (although I can if I have something I absolutely need). So, for me
>> > when I run verbose mode I simply get the DSO names in the callchains,
>> > but no arrows. I would assume, if I had a newer version of perf, that
>> > I would get no arrows for the symbols I want to annotate, given the
>> > behavior I'm seeing.
>> >
>> > Anything else I can try? Does anyone have any ideas about why this may
>> > be happening? Again, this is only happening for symbols in my static
>> > library, and I'm creating this library using:
>> >
>> > ar -cvrs $@ $(OBJS)
>> >
>> > (I'm not an expert on ar, so I could be doing this wrong?)
>> >
>> > On Thu, Apr 28, 2016 at 6:16 PM, Arnaldo Carvalho de Melo
>> >
>> > <arnaldo.melo@gmail.com> wrote:
>> >> Em Thu, Apr 28, 2016 at 01:44:16PM -0400, Mark Davis escreveu:
>> >>> Hello, I'm using perf record and perf report to profile an application
>> >>> which is made from a static (.a) library file that I made as well as a
>> >>> handful of C++ files that are not in the static library. They are all
>> >>> compiled with "-g -ggdb -fno-inline -O2 -fno-omit-frame-pointer" and
>> >>> all linked together to create the application.
>> >>>
>> >>> When I use perf report (in interactive/tui mode), when I annotate the
>> >>> symbols that are in the C++ files (not the static library), I see the
>> >>> source code intermingled with the assembly (this is what I want). I
>> >>> can do this at any layer of the call stack. However, when I drill into
>> >>> the symbols that are from the static library, I just see the assembly
>> >>> of the leaf symbol. I don't expect to see the source in this case, as
>> >>> the leaf symbol is a low-level library from libc; but, perf report
>> >>> won't let me drill into (i.e., annotate) any non-leaf symbols.
>> >>>
>> >>> Example:
>> >>> execute_native_thread_routine
>> >>> std::_Bind_simple<int (*(int, char**, Thread*))(int, char**,
>> >>> Thread*)>::operator()()
>> >>> main(int, char**)
>> >>> my_func_A
>> >>> my_func_B
>> >>> malloc
>> >>
>> >> What happens when you press 'V'? Here, say for this callchain:
>> >>
>> >> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.]
>> >> handlers_find>>
>> >>    - handlers_find
>> >>
>> >>       - 0.68% signal_handlers_foreach_matched_R
>> >>
>> >>            g_signal_handlers_disconnect_matched
>> >>            0x7f71fe809a80
>> >>            g_object_unref
>> >>
>> >>          - st_widget_get_theme_node
>> >>
>> >>               0.58% 0x7f71fe813429
>> >>
>> >> And I press V, I get:
>> >>
>> >> -    0.73%  gnome-shell      libgobject-2.0.so.0.4600.2          [.]
>> >> handlers_find>>
>> >>    -→handlers_find libgobject-2.0.so.0.4600.2
>> >>
>> >>       - 0.68% signal_handlers_foreach_matched_R
>> >>       libgobject-2.0.so.0.4600.2
>> >>
>> >>            g_signal_handlers_disconnect_matched
>> >>            libgobject-2.0.so.0.4600.2
>> >>            0x7f71fe809a80 libgnome-shell.so
>> >>
>> >>           →g_object_unref libgobject-2.0.so.0.4600.2
>> >>
>> >>          -→st_widget_get_theme_node libgnome-shell.so
>> >>
>> >>               0.58% 0x7f71fe813429 libgnome-shell.so
>> >>
>> >> See those arrows? They point to the places where you can annotate, the
>> >> others can't because they had no samples or were unresolved.
>> >>
>> >> With some changes I think we can show the ones without samples, but that
>> >> remains to be done.
>> >>
>> >> Also, this is with:
>> >>
>> >> [root@jouet ~]# perf --version
>> >> perf version 4.6.rc4.ga453697
>> >>
>> >> This feature was introduced in:
>> >>
>> >> commit 70e972788888315851a5c1b4982efbcafcd03b5b
>> >> Author: Arnaldo Carvalho de Melo <acme@redhat.com>
>> >> Date:   Thu Mar 19 16:07:21 2015 -0300
>> >>
>> >>     perf hists browser: Indicate which callchain entries are annotated
>> >>
>> >>     Now that we can annotate entries in a callchain, show which ones have
>> >>     an
>> >>     associated symbol and samples, by adding a right arrow just before
>> >>     the
>> >>     symbol name when in verbose mode.
>> >>
>> >>     To toggle verbose mode press 'V'.
>> >>
>> >> ---------------------------------
>> >>
>> >> Which is:
>> >>
>> >> [acme@jouet linux]$ git tag --contains
>> >> 70e972788888315851a5c1b4982efbcafcd03b5b | grep ^v4 | head -1 v4.1
>> >> [acme@jouet linux]$
>> >>
>> >> Thanks,
>> >>
>> >> - Arnaldo
>> >>
>> >>> When I annotate my_func_A and my_func_B, it just jumps to the
>> >>> annotation of malloc (I see this listed at the top of the annotation
>> >>> view). Note that my_func_A and my_func_B are both defined in the
>> >>> static library that I'm linking in. However, when I do a similar thing
>> >>> with a different stack where I'm annotating a function that's not from
>> >>> the static library (but in the regular C++ files), it works fine.
>> >>>
>> >>> Here's what I'm doing to create my static lib:
>> >>> ar -cvrs $@ $(OBJS)
>> >>>
>> >>>
>> >>> I did confirm with nm --debug-syms that my static lib has debugging
>> >>> symbols in it:
>> >>>
>> >>> U __assert_fail
>> >>> 0000000000000000 b .bss
>> >>> 0000000000000000 n .comment
>> >>> U __cxa_atexit
>> >>> 0000000000000000 d .data
>> >>> 0000000000000000 N .debug_abbrev
>> >>> 0000000000000000 N .debug_aranges
>> >>> 0000000000000000 N .debug_info
>> >>> 0000000000000000 N .debug_line
>> >>> 0000000000000000 N .debug_loc
>> >>> 0000000000000000 N .debug_ranges
>> >>> 0000000000000000 N .debug_str
>> >>> U __dso_handle
>> >>> 0000000000000000 r .eh_frame
>> >>> U exit
>> >>>
>> >>>
>> >>> How can I compile and link my application and configure perf report so
>> >>> I can drill in like this on functions defined in a static library with
>> >>> debug symbols?
>> >>>
>> >>> Thank you,
>> >>> Mark
>
>
>
> --
> Milian Wolff | milian.wolff@kdab.com | Software Engineer
> KDAB (Deutschland) GmbH&Co KG, a KDAB Group company
> Tel: +49-30-521325470
> KDAB - The Qt Experts