Re: debuginfod-based dwarf downloading, was Re: Wrong Perf Backtraces

[Jiri Olsa <jolsa@redhat.com>]

On Mon, Mar 30, 2020 at 05:05:34PM -0400, Frank Ch. Eigler wrote:
> Hi -
> 
> I was reminded by the thread that debuginfo downloading for perf is
> worth automating more.  Since perf uses elfutils, recently-added
> facilities to elfutils 0.178 (2019-11) already allow automated
> downloading from some perf code paths.  The following patch [2] builds
> a little bit on that by ensuring that the downloaded debuginfo is
> plopped into the $HOME/.debug/ hierarchy explicitly, and should cover
> a few more code paths.  Feel free to take / adapt this code as you
> like.

I think it's a good base, thanks a lot!

I made few comments before reading above,
feel free to post v2, if not we'll take over ;-)

thanks again,
jirka

> 
> debuginfod can also do source code downloading on the fly, but the
> perf/util/srccode.c machinery would need buildids passed, and I
> couldn't quickly figure it out.  (Sorry, I also couldn't quickly
> figure out the perf testsuite.)

np ;-)

> 
> [1] https://sourceware.org/elfutils/Debuginfod.html
> 
> [2]
> 
> Author: Frank Ch. Eigler <fche@redhat.com>
> Date:   Mon Mar 30 16:15:47 2020 -0400
> 
>     perf build-ids: fall back to debuginfod query if debuginfo not found
>     
>     During a perf-record, use the -ldebuginfod API to query a debuginfod
>     server, should the debug data not be found in the usual system
>     locations.  If successful, the usual $HOME/.debug dir is populated.
>     
>     Signed-off-by: Frank Ch. Eigler <fche@redhat.com>
> 
> diff --git a/tools/build/Makefile.feature b/tools/build/Makefile.feature
> index 574c2e0b9d20..51e051858d21 100644
> --- a/tools/build/Makefile.feature
> +++ b/tools/build/Makefile.feature
> @@ -48,6 +48,7 @@ FEATURE_TESTS_BASIC :=                  \
>          libelf-gelf_getnote             \
>          libelf-getshdrstrndx            \
>          libelf-mmap                     \
> +        libdebuginfod                   \

we put new feature checks which are not widely spread yet
and not vitaly needed into FEATURE_TESTS_EXTRA and call:

    $(call feature_check,libdebuginfod)

before you check the feature

>          libnuma                         \
>          numa_num_possible_cpus          \
>          libperl                         \
> @@ -114,6 +115,7 @@ FEATURE_DISPLAY ?=              \
>           libbfd                 \
>           libcap                 \
>           libelf                 \
> +         libdebuginfod          \

I dont see the libdebuginfod.c, did you forget to add it?

>           libnuma                \
>           numa_num_possible_cpus \
>           libperl                \
> diff --git a/tools/build/feature/Makefile b/tools/build/feature/Makefile
> index 7ac0d8088565..1109f5ec96f7 100644
> --- a/tools/build/feature/Makefile
> +++ b/tools/build/feature/Makefile
> @@ -26,6 +26,7 @@ FILES=                                          \
>           test-libelf-gelf_getnote.bin           \
>           test-libelf-getshdrstrndx.bin          \
>           test-libelf-mmap.bin                   \
> +         test-libdebuginfod.bin                 \
>           test-libnuma.bin                       \
>           test-numa_num_possible_cpus.bin        \
>           test-libperl.bin                       \
> @@ -155,6 +156,9 @@ endif
>  $(OUTPUT)test-libelf-getshdrstrndx.bin:
>  	$(BUILD) -lelf
>  
> +$(OUTPUT)test-libdebuginfod.bin:
> +	$(BUILD) -ldebuginfod
> +
>  $(OUTPUT)test-libnuma.bin:
>  	$(BUILD) -lnuma
>  
> diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
> index 80e55e796be9..15eeecf4ff17 100644
> --- a/tools/perf/Makefile.config
> +++ b/tools/perf/Makefile.config
> @@ -467,6 +467,11 @@ ifndef NO_LIBELF
>      CFLAGS += -DHAVE_ELF_GETSHDRSTRNDX_SUPPORT
>    endif
>  
> +  ifeq ($(feature-libdebuginfod), 1)
> +    CFLAGS += -DHAVE_DEBUGINFOD_SUPPORT
> +    EXTLIBS += -ldebuginfod
> +  endif
> +
>    ifndef NO_DWARF
>      ifeq ($(origin PERF_HAVE_DWARF_REGS), undefined)
>        msg := $(warning DWARF register mappings have not been defined for architecture $(SRCARCH), DWARF support disabled);
> diff --git a/tools/perf/util/build-id.c b/tools/perf/util/build-id.c
> index c076fc7fe025..a33d00837d81 100644
> --- a/tools/perf/util/build-id.c
> +++ b/tools/perf/util/build-id.c
> @@ -31,6 +31,10 @@
>  #include "probe-file.h"
>  #include "strlist.h"
>  
> +#if HAVE_DEBUGINFOD_SUPPORT

s/if/ifdef

> +#include <elfutils/debuginfod.h>
> +#endif
> +
>  #include <linux/ctype.h>
>  #include <linux/zalloc.h>
>  
> @@ -636,6 +640,21 @@ static char *build_id_cache__find_debug(const char *sbuild_id,
>  	if (realname && access(realname, R_OK))
>  		zfree(&realname);
>  	nsinfo__mountns_exit(&nsc);
> +
> +#if HAVE_DEBUGINFOD_SUPPORT

s/if/ifdef

> +        if (realname == NULL) {
> +                debuginfod_client* c = debuginfod_begin();
> +                if (c != NULL) {
> +                        int fd = debuginfod_find_debuginfo(c,
> +                                                           (const unsigned char*)sbuild_id, 0,
> +                                                           &realname);
> +                        if (fd >= 0)
> +                                close(fd); /* retaining reference by realname */
> +                        debuginfod_end(c);
> +                }
> +        }
> +#endif
> +
>  out:
>  	free(debugfile);
>  	return realname;